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Siunmary : To what kinds of events can a 
McCulloch—Pitts nerve net respond by firing 
a certain neuron? More generally, to what 
kinds of events can any finite automaton 
respond by assuming one of certain states? 
This memorandum is devoted to an elementary 
exposition of the problems and of results 
obtained on it during investigations in 
August 1951* 


REPRESENTATION OF EVENTS 
IN NERVE NETS AND FINITE AUTOMATA 
S. C. Kleene 


INTRODUCTION : 

1. Stimulus and Response : An organism or robot receives 
certain stimuli (via its sensory receptor organs) and performs 
certain actions (via its effector organs). To say that certain 
actions are a response to certain stimuli means, in the simplest 
case, that the actions are performed when those stimuli occur 
and not when they do not occur. 

Since both the stimuli and the actions may be very com¬ 
plicated, the relationship between the two is very complicated. 

In order to simplify our analysis, we may leave out of 
account the complexities of the response. To do this, we reason 
that any kind of stimulation, or briefly, any event which affects 
action, in the sense that according as the event occurs or does 
not, under some set of other circumstances held fixed, a differ¬ 
ent action ensues, must have a representation in the state of 
the organism or machine, after the event has occurred and prior 
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to the ensuing action (which action may depend on the occur¬ 
rence of many other events). 

We then ask what kinds of events are capable of being 
represented in the state of the organism or machine. 

We shall see later (Section 5»5) that there is no loss of 
generality in considering the representation, in the case of 
•nerve nets, to have the simple form of the firing (or sometimes 
the non-firing instead) at a certain time of a certain neuron. 

Por explaining response as due to stimulus, it would then 
remain to assemble the complicated molar response out of these 
molecular representations of molar stimuli. 

In this remaining problem, it could make a great difference 
what events are selected for molecular representation, as the 
abstract from experience which is to form the basis of action. 

However, we shall not enter into this here, except as it 
reflects on the problem of representing events; nor shall we 
enter Into the analogies between the analysis Just described 
and the psychological phenomena in which raw sense data lead 
through percepts and concepts to overt behavior. 

2. Nerve Nets and Behavior : McCulloch and Pitts (19^5) 
in their fundamental paper on the logical analysis of nervous 
activity formulated certain assumptions which we shall recapi¬ 
tulate below (Section }). 

These assumptions are an abstraction from the data which 
neurophysiology provides. The abstraction gives a model, in 
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terms of which it becomes an exact mathematical problem to see 
what kinds of behavior the model can explain. The question is 
left open how closely the model describes the activity of actual 
nerve nets} and some modifications in the assumptions lead to 
similar models. Neurophysiology does not currently say which 
of these models is most nearly correct—it is not plausible that 
any one of them fits exactly. It is noteworthy, however, that 
one of McCulloch and Pitts' results is that these several other 
models are capable of producing only the same behavior as the 
first one. 

Until neuro-physiology tells us more about the actual 
process, it is Instructive to see what behavior the model 
admits. Our results are to the effect that M it could be this 
way, and quite possibly the real process is significantly simi¬ 
lar to this.” Furthermore, such studies have applications in 
robotology, when we wish to describe on paper (or build in the 
metal, using elements which behave like McCulloch-Pitts neurons) 
a i»obot to behave in a pre—assigned manner. 

This study can be pursued on two levels, a strictly 
practical one and a theoretical one. On the former, we are 
concerned with constructing particular nerve nets to give par¬ 
ticular described behavior; in the latter, we develop general 
methods for constructing nets to give behavior, and investigate 
the limitations within which this is possible. 

This memorandum deals with studies on the second level, but 
actually the two are not clearly separated. The general methods 
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may be practical or suggest methods which are, and the inves¬ 
tigations of the limitations may contribute better understanding 
of the problems which are faced on the practical level. 

McCulloch and Pitts give such a theoretical investiga¬ 
tion, consisting of a theory for nerve nets without "circles" 
(Part II of their paper) and a theory for arbitrary nerve nets 
(Part III). The present memorandum is partly an exposition of 
the McCulloch—Pitts results; but we found the part of their 
paper which treats of arbitrary nets obscure; so we have pro¬ 
ceeded independently here. 

Under the McCulloch—Pitts assumption of the all-or-nothing 
character of a neuron's firing (which is close to the biological 
reality) and their assumption which quantizes time so that all 
neurons have their moments of possible firing in phase, a nerve 
net has the character of a digital automaton. Here we are using 
"digital" in contrast to "analog," in the sense familiar In 
connection with computing machines. 

It seems quite clear that many physical processes of 
control are partly analog in character. For example, the res¬ 
piratory cycle of activity can be controlled consciously(by 
nervous means, which are digital); but most of the time it is 
regulated by a nervous response in the respiratory center of 
the brain to the carbon—dioxide level in the blood (an analog 
quantity). 

Just as in mathematics continuous processes can be approxi¬ 
mated by discrete ones, it is plausible that any analog elements 
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in bodily control could be approximated in their effect by 
digital ones. Nevertheless, the analog or partly analog con¬ 
trols may remain the simplest and most efficient. 

One of the results of systematic theoretical investiga¬ 
tions of the potentialities of digital control might be to 
demonstrate that other principles, e.g., analog mechanisms or 
the introduction of random inputs, may be necessary to produce, 
or to produce economically, certain kinds of behavior. 

Another tacit assumption of the present mathematical theory 
is that there are no errors in the functioning of neurons; i.e., 
a given neuron fires at a given moment, if and only if it should 
under the McCulloch—Pitts rules. Of course, this is unrealistic, 
either for living neurons or for the equivalent units of a 
mechanical automaton. It seems natural, however, to build a 
theory of what happens assuming no malfunctioning. In this 
theory, we represent the occurrence of an event by the firing of 
a single neuron. Biologically, it is implausible that impor¬ 
tant information should ever be represented in an organism in 
this way. But by duplications of nets (many processes being 
carried out in parallel circuits), one could expect then to 
secure the same results with small probability of failure in 
nets constructed of fallible neurons. 

Returning to the formulation of the problem as given in 
Sect. 1, we shall now in Part I snow that all events of a 
certain class can be represented by the firing (or in some cases, 
the non—firing) of a certain neuron. The discussion of the 
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converse is left mainly to Part II, where we generalize to 
representability in any finite digital automaton. 

PART I — NERVE NETS : 

3. McCulloch—Pitts Nerve Nets : Under the assumptions of 
McCulloch and Pitts (19^3), a nerve cell or neuron consists of 
a soma , whence nerve fibers ( axons ) lead to one or more endbulba . 

A nerve net is an arrangement of a finite number of neurons, 
in which each endbulb of any neuron is adjacent to the soma of 
not more than one neuron (the same or another); the separating 
gap is a synapse . Each endbulb is either excitatory or inhibitory 
(not both). 

We call the neurons (zero or more) on which no endbulbs 
impinge input neurons ; the others, inner neurons . (McCulloch 
and Pitts say "peripheral afferent neurons" for the former, but ' 
it is convenient to have a shorter phrase. "Efferent neurons" 
might be used for the latter, but it is not clear to us that 
this is appropriate. (As the present paper is only a working 
paper, we welcome suggestions as to improvements in the termi¬ 
nology, ) 

At equally separated moments of time (which we take as the 
integers on a time scale, the same for all neurons In a given 
net), each neuron of the net is capable of firing or not firing 
(being quiet ) in an all-or-nothing manner. For an Input neuron, 
the firing or non—firing at any time _t is determined by condi¬ 
tions outside the net. One can suppose each is impinged upon 
by a sensory receptor organ, which under suitable conditions in 
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the environment causes the neuron to fire at time _t. For an 
inner neuron, the condition for firing at time Jt is that at least 
a certain number h (the threshold of that neuron) of the exci¬ 
tatory endbulbs, and none of the inhibitory endbulbs, synapsing 
on it belong to neurons which fired at time t—1. 

For illustration, consider the following nerve net, with 
input neurons J, K, L, M, and N and inner neuron P. Excitatory 
endbulbs are shown as dots, and inhibitory as circles. The 
threshold of P is 3 as shown by the figure on the triangle repre¬ 
senting its soma. The formula written below the net expresses 



J K L M N 


P.(t) =£(^-1) &. K(t-l) £ L(t-l) ^ H(t-l) &• ff('pi) 

Pig. 1 

in logical symbolism that neuron P fires at time t; |In the 
symbols, "£(£)"]» if and only if (in symbols, "EE") all of 
J, K, and L and none of M and N fire at time t—l.(" 2c” means 
"and," and " " means "not"). 

The method of nerve net construction illustrated In Fig. 1 
applies for any number > 1 of unnegated propositions (3 in 
Fig. l) and any number £ 0 of negated propositions (2 in Fig. l) 
combined conjunctively. 
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Two other nets (Figs. 2 and 3) illustrate additional methods 
which will be used in nerve net constructions in Sect. 5. 



Fig . 2 


Fig . 2 


Here "V" means "or" (in the non-exclusive sense). 


4. The Input to a Nerve Net ; Consider a nerve net with k 
input neurons . The input (or experience ) over all 

past time up to the present moment Inclusive can be represented 
by a table or matrix with k columns corresponding to the input 
neurons, and with rows corresponding to the moments counting 
backward from the present moment t m £. The positions are filled 
with 0's and l*s, where 0 is to stand for quiescence, and 1 for 
firing, of the neuron in question at the time in question. 

For example, with k • 2 the matrix might be as follows: 
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N t N 2 

1 0 

1 1 

0 1 

1 1 

Pig . _4 

The 1 in the first row and first column means that Ni fired 
at time £; the 0 in the first row and second column that N 2 did 
not fire at time £; the 1 in the second row and first column that 
Hi fired at time £-1; etc. 

If this table Is extended down infinitely, we ha¥e a repre¬ 
sentation of the Input, thought of as extending over all past 
time. The discussion whether we should think of past time as 
infinite will be left to the place where it becomes crucial 
(Sect. 6.1). For the purposes of Sect. 5 we need merely assume 
that it extends back In each case as far as the number of rows 
of the matrix being considered there. 

By an event we mean any property of the input. Thus, any 
subclass of the class of all the possible tables represents an 
event, which occurs when the table describing the actual input 
belongs to this subclass. In coin tossing or dice throwing, 
examples of events are "heads" or "eleven" (as sum of the 
numbers of Bpots on the uppermost faces of the two dice). Here 
examples are: (l) Ni fired at time £. (2) N 2 did not fire 


£ 

p—1 
£-2 

£-3 
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at time £, and N r fired at time £-1. (3) One of Ni and N 2 

fired at time £. (4) Ni and N 2 both fired at time £. (5) N 2 

fired at some time. (6) N 2 fired at every time except £. 

Of these, the (present and) past described by the table in 
Fig. 4 constitutes an occurrence of events (1), (2), (3)> and 
(5)> but not of (4), while we need to know the rest of the table 
to see whether it constitutes an occurrence of (6). 

5. Definite Events : 

5-l« " Definite events 11 defined : We shall first restrict 
ourselves to events whfch refer to a fixed period of time, con¬ 
sisting of the ^ (> 1) moments £—^-fl,...,£ ending with the 

✓ 

present. This means that in any table such as that of Fig. 4 

we consider only the uppermost rows; e.g., with ^ » 3s 

t Ni N 2 

£ 1 0 Ni (£) & 

£-111 & Ni(£-1) ir N 2 (£-l) 

£-2 0 1 2 - Mi(£-2 T&N?(£-2). 

Fig . 5 

The formula at the right expresses the same as is expressed by 
the table; i.e., it says that N* fires at time £ ("N t (p_)") and 
(" £c ") N 2 does not fire at time £ ("N 2 (jd) "), and Ni fires at 
time £-1 ("Nj (£-1)"), etc. 

We call an event referring to just these £ moments definite 
of length (or duration ) ][. With k input neurons, there are 
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exactly k£ entries In a table describing the input for these 

moments. Therefore, there are exactly 2^ possible such tables. 

k \ 

Therefore, there are exactly 2* definite events of length ^ 

since any particular event (of length with k input neurons) 

is obtained by saying which of the 2^ tables would constitute 

(if they represented the actual past) an occurrence of the everit. 

We call an event positive , if it only occurs when at least 

one input neuron fires during the period to which the event 

k£ , 

refers. There are 2 8 - definite positive events, since now 

we exclude as an occurrence of tne event that past described 
by the table of all 0’s. 

5*2 Definite positive events : 

Theorem 1. To each of the 2* definite positive 

events of length £ ( with k input neurons ), there is a nerve net 
having an inner neuron which fires at time £+2, if and only If 
the event occurs during time £-^+1 t£ £• 

This theorem, except for the remark that the "lag" can be 
held to 2, is given by McCulloch and Pitts (1945). 

Proof: To illustrate, say the event is one which occurs 
if and only if the pattern of firings over the past is represented 
either by the table of Pig. 5 or by the following table: 


t N 2 

£ 1 0 

£-1 1 0 

£-2 1 0 


n,(e)2.5?W 
^ n, <e-i)&h«Ie-i)' 
g. N.tE-sj&^TF n. 


Pl£. 6 
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That is, just these two (out of the 2 a ‘ 3 ■ 64) tables are to 
constitute an occurrence of the event. The event is described 
by the following logical formula: 

(ki (£) % EJTeJS- Nt (£-1) 8- N* (£-1) % 5* (E-a)] 

V (k. (E)^S^F&N,( g -i)g-g^E=r)2w,( £ ^)g. gTFgT 

Figure ]_ 

This formula is a "disjunction” having two "members" or "terms," 
each of which is a "conjunction" having six "members" or 
"factors." The two terms correspond to the tables of Figs. 5 
and 6, respectively. 

A nerve net which represents the event with lag 2 is con¬ 
structed as follows: 



Figure 8 
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Using the method illustrated in Pig. 3, 

n1( £ )== Ni(£- 1), Ng ( 2 .) = E* (£t-D> 

Ni(p) = Ni(p-fi), N*( £ )= N 2 (£-2) • 

Now using the device of Pig. 1, 

«1(2H)» Nt(£) i »I(2) gr si( e) 5 TTI 7 g. N Hz) 

= n,( e ) & SHeT^N'(E- 1 ) 2-FTTF^T^iialE-a': 

i.e.. Mi fires at time £+1 if and only if the past is described 
by the table of Pig. 5 (or the first conjunction in Pig. 7). 
Likewise, the firing of Ma at £+1 corresponds to the table of 
Fig. 6 (or the second conjunction in Pig. 7). Finally, by the 
method of Fig. 2, 

P(£+2)iE Mi(£+1) V &(£+!). 


Combining this with what has already been remarked, P fires at 
£4-2 if and only if the event occurs during time £-2 to £. 


The method of the illustration applies to every definite 
positive event which occurs for some one or more tables. By 
the restriction that the event be positive, each table must have 
at least one 1 in it, which assures the applicability of the 
device of Fig. 1. 

There remains the case of the event which never occurs. 

This is represented, e.g., by the following net: 



P never fires at time £+2 
(or in symbols, e.g., 
P(£+2) = Nt( £ ) & N t ( £ y}. 

Figure £ 
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50 Simpler nerve neta : While this proves the theorem, 
it is to be observed that often much simpler nets can be con¬ 
structed than that given by the above method of proving the 
theorem. 

Readers having technical acquaintance with symbolic logic 
will recognize that the construction used in proving the theorem 
corresponds to the principal disjunctive normal form of Hilbert— 
Ackermann (1928) which describes the event. In the illustration, 
the normal form is the formula of Pig. 7. Each of the tables 
which describe an occurrence of the event is represented by a 
conjunction or term in the normal form and is taken care of 
separately in building the nerve net. This makes the proof of 
the theorem simple, but the net complicated. 

Consider for example the event which is described by saying 
that the table must be of one of the two following forms, where 
either a 0 or a 1 can be supplied Independently for each blank 

n «i 


t N i N a 


t N i N a 


£ 

JET 1 

£-2 


0 £ — — 

- ° r £“1 ~ i ' 

1 £-2 — ~ 

[SPE7^ai(2-2)) V (jBr*l) • 


Figure 10 
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In terms of complete tables, this event would be expressed 
by a choice between 2 +2 —2^*40 tables, including that of Fig. 5 
as one of them. The principal disjunctive normal form would be 
a disjunction of 40 conjunctions. The simple formula shown in 
Fig. 10 which represents it is a disjunctive normal form (not a 
principal one). The event is represented in the sense of the 
theorem by the following net. 




In this net we show only N s as an input neuron, although 
we defined our events in terms of two input neurons Ni and N* 
in our illustrations. The net of Fig. 11 can constitute a part 
of a larger net having Nj also as Input neuron, entering in such 
a way that it has no endbulbs on any of the neurons shown in 
Fig. 11. The example illustrates that if we begin by defining 
events relative to a set of input neurons, we need 

actually use In our net constructions only those of 

whose firing or non—firing affects whether the event occurs. 

There is a corresponding treatment, with the same lag, for 
conjunctive normal forms. We begin by considering the following 
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Illustration, in which the normal form is a principal one with 
k - 5 and £ - 1. 


V p 



P(£f2) =[Ni(£)VN c (£)VN3(£j^[Ni(£)vI?l£TvNj{£T) # (fTTIT 

VN a ( 2 )V/N 3 ( P )] . 

Figure 12 

To see that this works, observe that we use Fig. 2 in obtaining 
M a , so that 

Mt(£fl)= N t (£) V Na(£)>/N3(£)j 
but Fig. 1 to obtain M a and M 3 , so that 

M*(£H)=fTTFA n s ( £ ) ^ n 3 ( £ ), Ms(e+i)= Ni( £ ) 8t 

UTeT %, STTIT- 

Hence 

Mg (£+1 ) = N 1 (£)V^2T£) V N 3 (£) , M 3 (£+1 ) = N 1 (£) V Ng (£) V N 3 (£) . 
Also, we used Fig. 1 to obtain P, so that 

p(e+2) = m,( £ +i)& UTjwTT & HITEnT- 

Substituting our formulas for Mi(£+l), rt*(£+l) and M 3 (p+I)~ 
in the latter gives the formula for P(£+2) in the figure. 
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This method of treating a principal conjunctive normal form 
depends on that disjunction which has no negated propositions’ 
being one of the factors; but it must always be for a positive 
event, since otherwise the falsity of all the elementary pro¬ 
positions would make every term of the conjunction true. 

If the principal disjunctive normal form has n terms, the • 
principal conjunctive normal form has 20 -n factors, and vice 
versa (so the longer one form is, the shorter the other). We 
see why this is so in our illustration thus (omitting "(p)" after 
each ”N" for brevity). 

[n, 

Under the bar in the second expression we have the principal dis¬ 
junctive normal form of the negation of the first expression, so 
the last expression (which is the principal disjunctive normal 
form of the original expression) is obtained by combining dis¬ 
junctively those 5 of the 8 elementary conjunctions which do not 
appear in the second expression. 

When an event can be represented by a conjunctive normal 
form other than the principal one, a corresponding simplifica¬ 
tion can be made In the net construction Just as In the case of 
disjunctive normal forms. 

By using a normal form of either kind, we have held the 
lag s in the representation to 2. 
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Now it may happen that the most compact formula we have 
at hand to represent a given definite positive event is not a 
normal form. Then we can construct a nerve net of exactly 
corresponding structure and complexity, if we accept a greater 
lag £. In fact, the lag will be exactly the '’depth 1 ’ (or number 
of "layers") in the formula in terms of the operations 2. and V* 
We shall see this in Sect. 5-4 (Theorem 2). 

For some events, of course, a lag of 1 suffices (or even 
a lag of 0 or —1 or -2, etc., if respectively the event speci¬ 
fies nothing about the firings at time £ ox times £-1, £ or 
times £-2, £-1, £, etc. Reduction of the lag below 2 is not 
possible in general (with the assumed kind of neuron). A 
counterexample is the event Ni (j>) & (N* (£) y TTs To repre¬ 

sent this with lag 1, the net would have to consist of the 
representing neuron P with endbulbs belonging directly to Nx, 

N a , and N 3 . One readily sees that no such net represents the 
event in question. 

To hold the lag to 2 in all cases by use of a normal form, 
we may be obliged to have a very large number of endbulbs 
synapsing on a given soma, or of axons emerging from a given 
neuron. Biologically there are limitations. A relatively small 
increase in the lag will cut tnese numbers down. For example, 
a soma with 10^ excitatory endbulbs synapsing on it is replace¬ 
able with an increase of only 2 in the lag by a net made up so 
that only 10* endbulbs synapse on each soma (but, of course, now 
a large number of neurons are necessary). 
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5.4 Definite events In general : 

Corollary . To each of the 2 definite non—positive 

events of duration £ with k input neurons, there is a nerve net 
having an inner neuron which does not fire at time £+2, if and 
only if the event occurs in time £-^+1 to £. 

Proof . Denote the event by E, and by E the complementary 
event or negation of E, which occurs exactly if E doeB not 
occur. The set of tables, one of which the past must fit if E 
occurs, is the complement (in the set of all 2^ k by £ tables) 
of the set, one of which the past must fit if E occurs. 

Now E is positive, so by the theorem (Sect. 5*2), there 
are a net and neuron which represent E by firing that neuron 
at £+2, and therefore represent E by not firing the neuron at 
£+ 2 . 

Theorem 2 . Consider any logical expression E in tferms of 
v, _ and propositions N^(.t) (1 < i^ < k, £~^+l < _t < £) 
describing a definite event E of length ^ with k input neurons . 
Then there is a nerve net of corresponding structure which 
represents E by firing or by not firing, according as E is 
positive or non-positive, a certain neuron at time £+s, where 
b is the depth of E in terms of and V only . 

Proof ; It will be convenient to assume there are no double 
negations in E, as can be arranged by use of the law of double 
negation E*R. (This does not change the depth.) 

First we give the treatment for the least depth 1. For 
convenience we take ^ to be 1, writing "Hi,” "U 2 , M etc., for 
m Ei(jb)," "Ha ( 2 .) •" But for ^ > 1 we would merely need to use the 
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method of Pig. 3 to introduce neurone whose firing at t » £ 
represents the firing of the various input neurons at the earlier 
times. 

By an elementary conjunction (elementary disjunction ), we 
mean a conjunction of one or more factors (terms) each of which 
is an elementary proposition Ni, N®, etc., or a negated elemen¬ 
tary proposition ]7 a , etc. (By allowing one factor or term, 
a single proposition or negated proposition can be considered 
as either a conjunction or a disjunction here.) 

Now we have four basic oases to treat. 

Case 1 : An elementary conjunction containing at least one 
unnegated factor, e.g., Ni 2- H» 8-Na iff. St'lg. The event is 
then positive; so we want to represent it by the firing of a 
neuron at time jh- 1. Use Pig. 1 to obtain this neuron. 

Case 2 : An elementary conjunction containing only negated 
factors, e.g., Hi 2- F 2 The event is non-positive. But 

now its negation Ni ^ |? 2 ^ n 3 is positive. The latter Is equi—> 
valent to N| V Ji® Vlis. Use the method of Pig. 2 to represent 
this by a neuron firing at £+1; this neuron then represents 
the original event by non—firing at £+1, as we wished to have it 
represented. 

Case 3 * An elementary disjunction containing at least one 
negated term, e.g., R x V ST® V T7 3 V N 4 V N^. The event is non- 
positive. But its negation Hi V fi®" s/ H 3 V ]i* V is positive, 
and the latter is equivalent to N x N® £ N 3 £ Use 

Fig. 1 to represent the latter by firing at £flj then the origi¬ 
nal event is represented by non—firing at g*l. 



RM—704 
Page 21 


Case 4 : An elementary disjunction containing only 
unnegated terms, e.g. , N»VN S VN 3 . The event is positive. 

Use the method of Fig. 2 to represent It by firing at £+1. 

The cases are mutually exclusive, except that a single 
unnegated proposition N can be considered as under either 
Case 1 or Case 4, and a single negated proposition IT as under 
either Case 2 or Case 3* But for one input only (which must 
be unnegated for Fig. l). Figs. 1 and 2 coincide; so the treat¬ 
ment Is actually the same. (Indeed, for N or | It is only to 
have an inner neuron which represents them that any treatment 
is necessary; otherwise, we could consider them as representing 
themselves at time £.) 

The treatment of a formula with depth > 1 requires only 
iteration of the processes used in the four basic oases. 

It will suffice to illustrate by a complicated example, in 
which the depth is 4: 


4 

3 






2 

1 


(For handy reference we took an example with all N's different, 
but they could be identified in any combinations.) The under¬ 
lines indicate the parts of various depths. Also N a , IT* and 
are parts of depth 1, which we could treat as "degenerate" 
elementary disjunctions, but there is no need to consider them 
thus here. 
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First we apply Case 4 to obtain a neuron Mi which repre¬ 
sents NiVNg by firing at £+1# and one M 3 which represents Ng v/N^ 
likewise; and Case 3 to obtain one M a which represents 
by non-firing at £+ 1 . 

Next consider the two parts of depth 2: 

(NiVN*) S5 

Replacing N i VNa (i.e., Ni (£) \ZN 2 (£)) by its equivalent Mi(£tl), 
and NgVNy by its equivalent M a (£+1 j (since it is the non- 
firing of M 2 at £4-1 which represents it), these become, respec¬ 
tively, in terms of inputs from Mi and Mg at £tl and of N 3 , N« 
and at £: 

m.<e+i)&!Me)S.E^TF, «»li +1 ) • 

(As the inputs are no longer all at the same time £, we show 
the times.) The left one we treat by Case 1 to obtain a neuron 
Li which represents it by firing at £+ 2 , and the right one by 
Case 2 to obtain one La which represents it by non—firing at 
£+ 2 . 

Next consider: 

|(N»VNp)&n4e 3 v (?5^^ V H7)] • 

Replacing the first term by its equivalent L x (£f2), and the 
second by its equivalent La(£+2) (since E 5 & (£16 ^ E 7 ) is equi¬ 
valent to La(£+ 2 )), we obtain 

Li(£f2)V L a (£+2). 

This we treat by Case 4 to obtain a neuron Q which represents it 
by firing at £+ 3 . 
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Finally, consider the entire expression. Replacing the 
two factors by their respective equivalents, we obtain 

a<EO)a-i»'(£iTT. 

which we treat by Case 1 to obtain a neuron P which represents 
it by firing at p+4. 

Incidentally, .we have discovered in the process that the 
event is positive—we did not need to take the trouble of set¬ 
tling which it was at the beginning. 

Both Theorem 1 and its corollary are corollaries of the 
present theorem, and the nerve net constructions in Sects. 5*2 
and 50 for disjunctive and conjunctive normal forms are by 
the present method; so we might have given Theorem 2 first. 

Other logical operations which might be used in defining 
events are definable in terms of and ; e.g., (F-^G)*FVO 

is read "implies" or "if ..., then ..."), and (F5G) S 
((FH>G) % (G-*F)). 

Summarizing, given any description, in words capable of 
being translated into logical symbolism, of a definite event, 
we have the means for constructing a nerve net to represent it 
of exactly corresponding complexity. So the theory of nerve net 
construction for definite events is as practical as one could 
ask. The lag can always be held to 2 for a given event, but 
sometimes a greater lag will correspond to a simpler description 
of the event, and give us a simpler net. 

There may in special cases be simpler nets than those given 
by the method of proof of Theorem 2. We see this by considering 
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the condition for firing of any inner neuron of thereshold h 
at time t, which is that some set of neurons having a number 
> h of endbulbs on it, and none of the neuronB having inhibitory 
endbulbs on it, fire at time t—1. The condition for not firing 
is dual to this. For example, the nets below represent with 
lag only 1 the events described below them uBing expressions 
of greater depth (the upper by firing, the lower by non—firing): 

t t 



[(Ni& N a ) v (N x &N 3 ) V(Na&N S j] £ 

[ 5 iV 5 e)S- (£» vn; 3 ) l (S* vf 3 )] v N4 v/n 5 


V P 



(Ni VN?)&Ns^54 


(EifiE*) v N* 


Figure 12 glgigO: 14 

(Compare Fig. 14 with the treatment of (N* VN*)^ Nsj^E* in the 
long example for Theorem 2.) We have not undertaken to study 
how much net simplification might be gained by attempting to use 
this method systematically with the help of appropriate logical 
transformations. 


5*5 Representation of eventB in general : We can now pro^je 
the remark we made in Sect. 1 that there is no loss of generality 
in considering the representation of an event to consist of the 
firing or the non—firing (as appropriate) of a single neuron. 
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By saying that an event (occurring over a time ending 
with the moment p) Is represented In a nerve net at a certain 
time £fs (s >0), we mean that some property of the state of 
the net at time £+8 is equivalent to the event having occurred 
ending at time £| l.e., according as the event did or did not 
occur, the net will or will not have that property. 

But what happened at times < £ can only affect the state 
of the net at times £f£ for s > 0 via the state of the net at 
time £. 

Say besides the k input neurons there are m Inner neurons. 
The state of the net at time £ consists of the condition 
(firing or non-firing) of each of the m+k neurons. Thus, there 

are exactly 2- possible states at time £. There are 2* 

properties of the state of the net at time £. Any event ending 
at time £ which can be represented at time £H3 is thus equivalent 
to one of these 2“ properties of the state at time £. 

But for each of these, by applying the method of proof of 
Theorem 1 or its corollary# or of Theorem 2, to all the m+k 
neurons (instead of only the input ones) and to only the moment 
£ (instead of the interval e-£ + 1 to £), we can add additional 
neurons to get a neuron P which will fire or not fire (according 
as the property of the m+k neurons at time £ is positive or not) 
at time £+2, if and only if the nvfk neurons fulfill the property 
at time £i and hence, if and only if the event in question 
(referring to input neurons and ending at time £) occurred. 

Incidentally, we have not made any assumption here whether 
the event in question is definite or not. • 
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In organizing a complex of stimuli Into a complex of 
responses (Sect. 1) as economically as possible, it is to be 
expected that the representation of events will not always be 
compressed into the form of the firing or non—firing of a single 
neuron. 

5.6 Nerve nets without circles : A circle (of length c) 
in a nerve net is a set of distinct neurons Ni,.,.,N such that 

has an endbulb on (,i » 1,...,£—l) and N q has one on N x . 

' mm ,r * 

Theorem 3 ‘ Given any nerve net without circles and any 
inner neuron N in that net, the firing (non—firing) of that 
neuron at time £+-1 is equivalent to the occurrence of a definite 
positive (non-posltlvc) event ending at time £. 

This theorem is stated for positive events by McCulloch 
and Pitts (1943). 

Proof . Whether N fires at £tl Is completely determined by 

i t 

the firing or non—firing at £ of those neurons Ni,...,N having 

i i ~ 

endbulbs on N. Consider those of Nx,...,N r which are inner 
neurons, and repeat the argument. Since there are no circles, 
any chain of neurons each Impinged upon by an endbulb of the 
preceding must terminate. Let ^+1« the length of any longest 
such chain; a longest must exist since there are finitely many 
such chains, and ^ £ 1 since N is inner. Then the process ter¬ 
minates after £ steps. Thus, the firing or non—firing of N at 
£+1 is completely determined by the firing or non—firing of 
certain input neurons at times 2“£ +1 to £; i.e., it is equiva¬ 
lent to a definite event of duration The event must be 
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positive, as firing can only be propagated but not originated 
under the law for an inner neuron’s firing (Sect. 3 ). 

Then of course N's non—firing at time £4-1 is equivalent to 
the complementary event, which is non-positive. 

(Any definite event is expressible by a logical formula, 
e.g., by a principal disjunctive normal form as in Sect. 5*2. 

So £ Priori there is a formula. By utilizing the condition 
for firing at each synapse, which we formulated in words in 
the last paragraph of Sect. 5*^ and could have in symbols, one 
can, of course, build up a formula in £ Btages, aB McCulloch— 
Pitts indicate.) 

Corollary : For a net without circles, any event ending 
at time £ which can be represented by the firing (non-firing) 
of a given inner neuron N at a certain time £+si («s > 2) is 
definite and positive (definite and non-positive) . 

Proof : For by the theorem, the condition for the firing 
of N at time £+jj is the occurrence of a definite positive event 
ending at time £+s—1. But since by hopothesis, N's firing 
represents an event ending at time £, the input over time 
£+1,... ,£+a—1 cannot affect whether the aforesaid definite posi¬ 
tive event occurs ending at time £+£—1. So in fact that definite 
positive event can be taken to refer only to a time ending at £. 

This corollary constitutes the converse of Theorem 1 and 
corollary (or Theorem 2). Likewise, any event ending at £ 
represented by a state, or a property of the state, at a time 
£+£ of a net (s£o) without circles, is definite. 
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6. Indefinite Events—Preliminaries ; 

6.1 Some examples : Consider the following nerve net 
(with a circle of length 1 consisting of M). 



Figure 15 

If at some time t £ £ the neuron N fires* then the firing 
of M at time £tl (and at every subsequent time) will follow. 

In symbols, 

(Et) t < gNfe)-> M(^fl) 

("(Et) t . "‘Is read "there exists a t ^ £ n ). But we do not 

have equivalence ("^") instead of merely implication ("—>"), If 
past time Is taken as Infinite, since the firing of M at time 
£+1 can also be explained by firing of M at every past moment, 
without N having ever fired. 

Similarly, the net 



Figure 16 

will only fire at time £+-1 if N has fired at all past times; 
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in symbols 

(i) i< E- i(E) «—- (£tl) 


(read "(t;). " as "for all Jt £ £ n )i but not conversely, for 

M may fail to fire at time £fl when (.t) t ^ ^J<{£) is true, by 
falling to fire over all past time. 

(Et)^ ^ ^N(£) and )t <£ .gN(£) are simple examples of 
events not referring to a definite period of past time; and we 
see that, under the assumption that past time is infinite, the 
nets shown do not represent them, by firing at time £fl, in the 
sense of equivalence (the first is represented in the Bense of 
"necessity" only, the second "sufficiency" only). 

If we attempt to represent the former by non—firing, we 
have a net 



for which (Et) t ^ -> Mlg+T), but not conversely. 

The difficulties encountered in these three examples are 
not escapable by using other nets to represent the events, or 
in other examples of indefinite events, but constitute the 
general rule for indefinite events. We shall show In Appendix 1 
(Theorem 7, to be read after Part II) that, under the assumption 
of an Infinite past, an event can be represented (in the sense 
of equivalence) by the firing or by the non—firing of a certain 
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inner neuron at time £+•£» (for any fixed s ^ l), only if the 
event is definite. For any net and inner neuron and fj, and 
any indefinite event, it must either be possible to have a 
past for which the event does not occur and the neuron fires, 
or one for which the event does occur and the neuron does 
not fire, or both. 

Of course, any living organism or actually constructed 
robot has only a finite past. The mentioned result shows that 
now we must take this into account; otherwise, we might have 
been tempted to use the fiction of an infinite past to simplify . 
the theory. 

6.2 Initiation ; Accordingly, let us assume that the past 
for our nerve nets goes back from £ (the present) a certain 
finite time only, the first moment of which shall be 1 on our 
time scale, (We find it more convenient notationally to call 
the first moment t ■> 1 than _t <* 0, but if we think of each 
positive integer _t as referring to the final instant of a unit 
interval, this does make time start from 0.) 

In seeking to represent events, we shall now assume the 
right not only to construct the nerve nets as we please, but 
also to fix the state (firing or non—firing) of each inner neuron 
at time 1. That is, we study representation of events in nerve 
nets started with a given internal state at the initial moment 1. 

The range of the time variables in our logical formulas 
shall now be the Integers from 1 forward, and this shall be the 
only part of the past we talk about except when we make it plain 
we intend otherwise. 
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Now the nerve net of Fig. 15» started at t « 1 with M 

quiet, represents by the firing of M at jh- 1 the event 

(Et). ✓ Jl{t); and net of Fi 8« 16, started with M firing, 

— ISE 

represents likewise (_t ) t ^ . That of Fig. 17 started 

with M firing represents (Et) t< - _N(.t) by the non—firing of M 
at £+1. Thus, the two nets of Fig. 15 and 17 are able to 
remember if N has fired since their beginning by changing M from 
the state it had initially; while the net of Fig. 16 is able to 
recognize that N has never failed to fire by preserving M in 
the state it had originally, as Householder and Landahl (19^5) 
have commented (p. 109 ). 

To represent (t) t ^ ^(t) either by the firing or the 


non—firing of a neuron in a net with only N as input neuron, 
at least one inner neuron must be fired initially. For were 
all inner neurons quiet at time t * 2, then in case Njl) 

(i.e., if the input neuron N does not fire at time t ■ 1), all 
neurons would be quiet at _t * 2. So the state of the inner 
neurons at t *= 2 would then be indistinguishable from that at 
Jb «= 1. Hence, the net at any time £+1 3 would have the same 

state whether the past is 


which makes (,t) t 


t i 2 3 4 

N(t) 0111 
< ^(t.) false, or 


9 


t 1 2 3 4 • • • 

N(t) 1 1 1 1 ..., 
which makes (t)^ ^ ) true. 
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The case in which all inner neurons are initially quiet 
is natural neurologically; the other case leaves it unexplained 
how the firing of certain inner neurons is to be produced at 
_t = 1. Of course, a natural explanation would be available 
by setting the time origin back, if the initial state in ques¬ 
tion is one which could be brought about by a suitable pattern 
of firings of the input neurons over a finite preceding time 
at the beginning of which all the inner neurone are quiet. But 
this is not so; e.g., in the case of the simple net for 
(t) t < ^M(t) (Pig. 16 ). 

Although (t), . N(t) cannot be expressed (for N as sole 

" “ i 2- 

input neuron) without having an Initially fired inner neuron, 
( 5U) U < JK<H) fc (l)g < JTI7 2c (t) H with two input 

neurons K and N can be, by the following net, in which P fires 
at £*2 if and only if the event occurs. 



The neurons K, Li and L 2 act as a starting circuit, which can 
only be used once, for the generality circuit N, M and P. 

The device with a modification is general. Suppose we have 
given a net in which an event is represented by the firing of a 
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certain inner neuron at time g+s if certain inner neurons are 
initially fired. 

First, let us add a starting circuit K, L| and La from a 
new input neuron K, with axons from its neuron L* leading to all 
the same neurons and with the same kinds of respective endbulbs 
as the axons from each of the inner neurons which were initially 
fired in the given net. 

Furthermore, each input neuron N of the original net we 

i 

now make an inner neuron N with a threshold of 2, and we 
insert new neurons N and R, the former taking over the role of 
the original N as input neuron, and the latter an inner neuron 
as shown. The heavy line stands for the axons which lead from 
the original N. 



This accomplishes the double purpose of lagging the input from 
N by 1 and of blocking it for moments prior to the first moment u 
at which K is fired. 

Now with all inner neurons initially quiet, no neuron 
except N can fire until the first moment u at which K fires. 
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Then at moment u+1 the neuron Li takes over the role of the 

t 

original initially fired inner neurons, while N carries the 
input of N at u. At every subsequent moment u+v+1 (v £ l), 
all of the original neurons (counting N as the original N) 
will behave as they formerly would have at time v+1, if the 
present input over u,...,u+v had been the input over time 
1,* *•,v+1* 

So the output neuron will fire at g+s+l, if and only if 
the event now occurs relative to _t *» u instead of to t - lj 
i.e., we have a representation of the event redefined to refer 
not to the whole past but to the past beginning with £ ■ u, 
and with an increase of 1 in the lag in the representation. 

Now if it is assumed that there are conditions in the 
environment which would continually stimulate K to fire, or 
that at least such a condition exists at t ■ 1, then our net 
will represent the event relative again to the whole past 
(since now u * l). Thus, we are provided with a "natural" way 
of getting a representation of any event, referring to the input 
neurons besides K, which could be represented "unnaturally" by 
the firing of a neuron in a net started with some initially 
fired input neurons. 

Here "natural" means only that we do not need to go out¬ 
side the McCulloch—PittB laws of neural behavior to fire some 
inner neurons at t ■ lj but the starting circuit K, Li, Lp, and 

1 

the blocking circuit K, R, N are not thought of as plausible 
mechanisms biologically. However, our first aim is to see what 
is at all possible, and one can then seek other and perhaps more 
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natural ways for accomplishing the same. 

This argument that by assuming an Initially stimulated 
input neuron K we can avoid having to have any initially fired 
inner neurons applies only to representation of events by 
firing a neuron, if for representation by non—firing at £fs+l 
one wishes that the output neuron fire at times l,...,s+l. 

The question involved will be analyzed in Sect. 6 . 3 . 

The same construction but omitting the K and La of the 

» 

starting circuit and the delaying—blocking neurons R and N , 
and firing L t initially, shows that it is always possible, if 
we are to use initially stimulated inner neurons, to hold the 
number of them to 1, without any increase in the lag. This 
again is for the case of representation by .firing. For repre¬ 
sentation by non—firing, the situation is slightly more compli¬ 
cated, and we shall not go into it. 

As stated, we ordinarily consider nets only for a speci¬ 
fied initial state of the inner neurons. However, McCulloch 
and Pitts consider the problem of "solving” nets with their 
initial condition unspecified. To "solve" for a given inner 
neuron P, say at time £fl, means then to find for what inputs 
over time 1 ,...,£, and what initial states of the inner neurons, 

P will fire at t - £+1. Now in the following net, the necessary 
and sufficient condition that P fire at £+1 is that N fire at 
all times < £ and both Mi and M a fire at time t - 1. 

P(£fl) = 

(t) t< 2 N(t)S,M,( 1 )^K 8 ( 1 ) 

Figure 20 
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This seems to be counterexample to the formula next after 
( 9 ) on p. 126 of McCulloch—Pitts (1943), the proof of which we 
did not follow; for if we understand the formula correctly, it 
implies that the condition for firing should only require the 
existence of one (suitably chosen) neuron known to fire initially. 
In this example we cannot conclude that P fii*es at £tl on the 
basis of any information which tells us only that one of the 
neurons fires at t • 1. (Our 1 seems to be their 0.) This 
apparent counterexample has discouraged us from further attempts 
to decipher Part III of the McCulloch—Pitts 1943* 

6.3 Definite events reconsidered ; Now that we have 
Introduced the assumption that the past for a nerve net is 
finite, we must reexamine the treatment of definite events which 
was given in Sect. 5. 

What happens now when £ < £; i.e., when the period of time 
to which the event is supposed to refer extends back to before 
the moment _t « 1? 

Generally, one may suppose that the durations £ of defi¬ 
nite events which are significant for an organism will be small 
in relation to the age £ of the organism at which the event Is 
significant. 

This, however, does not enable us to dismiss the problem. 

For to make the theory of nerve net control accurate, we should, 
for each definite event considered, either (a) show that an 
’'hallucination" that the event has occurred arising during the 
first £-1+13 moments of life could not have any serious effect 
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on behavior, or (b) provide against the occurrence of such an 
hallucination in the first 2H-+S. m oments. 

When we use definite events to build indefinite ones, 
e.g., the event consisting of a certain definite event having 
occurred ending at some time £ <£ £ (briefly, the memory of the 
definite event having occurred), such an hallucination could 
conceivably have a long-term effect, even if it has no immediate 
effect on behavior. 

The solution by (a) is, of course, outside the present 
investigation, and belongs rather to the full problem of orga¬ 
nizing stimuli into responses (Sect. 1). 

For organisms, the picture of the nervous system coming 
into activity in toto at a fixed moment it ■ 1 is implausible. 

But this means only that organisms (at least those which sur¬ 
vive) do solve the problem for their process of coming into 
activity. 

For machines, it is familiar that starting difficulties 
may have to be taken into account by the engineer. 

To take a fictitious illustration, consider the case of 
the "rat satellite robots" for the Tuvian Navy. A rat satellite 
robot is Intended to go about a ship, and whenever after three 
hours (= ^—1 moments) it has not smelled a rat, and at the next 
moment (the th) land is in sight, the robot abandons ship. 

The robots were ordered from RAND and were built by the 
Robotry Section from blueprints prepared by the Logicians Group 
on the basis of the theory in Sect. 5 above, with two input 
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neurons, Ni which is fired by the smell of a rat, and Ng,which 
is fired by the sight of land. The inner neuron P, which fires 
at time £+2 if the event "no rat smelled for ^—1 moments, land 
seen at the th moment" has occurred during the time £-*^+1 to £, 
was connected to an effector mechanism for abandoning ship at 
time £+3* 

Suitable ceremonies were scheduled for the occasion of 
their installation in the harbor of the Tuvian Naval Base. 

When the occasion arrived, they were placed on board the 
ships, and their batteries were connected up supplying power 
for operating the nerve nets and effector mechanisms. But three 
moments later. Just as the Tuvian Grand Admiral was congratula¬ 
ting the RAND delegate, all the robots went overboard! 

Proceeding to details, it is, of course, a matter of 
definition how we shall interpret "events of duration when 
p < 2;. But whatever definition is adopted, we must keep the 
facts about nerve net behavior straight. 

We 3hall (as best suits our present purpose, which is to 
lay a firm basis for the theory in Sect. 7) say that an event E 
of duration £ can only have occured ending at £ when £ ^ 

Then, of course, the logical formulas we have used to 
represent the events in Sect. 5 are not complete. If E x is the 
formula which described a definite event of length ^ there, 
the formula E which describes it fully now is E 1 The 
negation E of this is ^iV£ < ^ , while the formula for the 
"complementary" event of duration £ is rather Ex^£ > Thus, 
some care is now necessary in connection with the operation of 
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negation. The theory In Sect. 5 is applicable to the part of 
the formula which does not give the time reference; i.e., the 
relationships studied there apply to the Ei and Ei. 

Now consider.a nerve net as constructed for Theorem 1 in 
Sect. 5.2 to represent an event occupying the time Interval 
Ji-2+1 to £ of length £ by firing a certain inner neuron (the 
"output neuron") at time £f2. 

Using this net under the restriction now that the life of 
the net starts with a certain moment t •* 1, and under the sti¬ 
pulation that at that moment all Inner neurons are quiet, it is 
clear that the output neuron will fire at any time £+2 for 
£ > ^ correctly; i.e., if and only if the event occurrrd in the 
time £—^+1 to £. 

But the net might also fire at a time < ^+2; namely, thiB 
could happen if and only if the event is such that our present 
initial condition of the inner neurons (all quiet) is one which 
could also take place in Sect. 5-2 at some moment m where £-£+2 
^ m < £+1 for some occurrence of the event in £~^+l to £. 

So assume (in the context of Sect. 5.2) that we have an 
occurrence of the event in the course of which all inner neurons 
are quiet at t. * m. 

The state of the inner neurons at time m would then have to 
be the same (i.e., all quiet) if the table describing the 
past is altered to show only 0's for all input neurons at all 
times t < m. For from a past consisting entirely of non—firings 
prior to m, no firing of any Inner neuron can be produced at 


time m. 
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Since the state of the inner neurons is unchanged at 
time m, and the inputs for t » m,...,£ are unchanged, the 
output neuron still fires at t « £+2. So by Theorem 1, the 
event still does occur in time j^-^+1 to £; i.e., we now have an 
instance of the event occurring in which no Input neurons were 
fired in £~^+l to ro-1; In particular (since m > £-^+2), the 
event can occur with input 0 on all its input neurons at its 
first moment £~^+l. 

Conversely, if this is the case, the output neuron will fire 
now at time £+1, if in times 1,...,£—1 the inputs are what they 
could be in Sect. 5.2 for the momenta £--^+2 to £ of an occurrence 
of the event with only 0’s for £-^+1. 

Call a definite event of length ^ prepositive if the event 

can occur only when some input neuron fires in its first moment 

k£ 

£-2+1 i i.e., the selections from among the 2* possible k x £ 
tables which describe occurrences of the event all have at least 
one 1 in their bottom row. The prepositive events are a subclass 
of the positive events. 

Now we have shown that a necessary and sufficient condi¬ 
tion that no '’hallucination" be possible (in the sense of the 
output neuron’s firing at a time t when the event has not occurred 
ending at time b-2) is that the event be prepositive. 

We gave the reasoning for the case in which the event is 
to be represented by firing of a neuron at time £+2 (corres¬ 
ponding to Theorem 1), but it applies equally well to the other 
cases in Sect. 5; i.e., to representation by firing at £f£ for 
any given s. > 1, or to representation by non—firing at £+s 
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(then hallucinations are always possible, since the event is 
necessarily non-positive by Theorem 3 or its corollary), or 
to representations by a property of the state at a certain 
time p+s (a. > 0)* (In giving the sufficiency proof, we write 
the Inequality on m now p--£+2 £ m <; j>+s, and change the input 
for t max(nv-l,p). ) 

Most but not all events we may wish to consider will be 
prepositive. 

The analysis is valid for any net which operates correctly 
when £ > ^, whether constructed as in Sect. 5 or not, and 
started now with all inner neurons quiet. 

If a positive event of length ^ is not prepositive, we can 
build a net which represents it by firing a neuron P at time 
£+2 ( or p+s for some s > l), if this net is started at t ■ 1 
with one of its inner neurons fired (but all others quiet), as 
follows. We simply take an inner neuron L as in Figure 21, 
Initially fired (as the 'V indicates). 



Figure 21 

We then treat this as though it were an additional input neuron, 
required to fire at t » p—2(+l, in applying the method of net 
construction of Sect. 5* (Of course, then more than one axon 
may be required from L to other neurons.) 
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This device also makes it possible to represent non- 
positive events of length £ by firing of a neuron at time g+2 
(or for some s > 1); in Sect. 5 they would have to be 
represented by non—firing. 

Another device for fixing a net, constructed as in Sect. 5 
to represent a positive but not prepositive event (then ^ > 2), 
so that no hallucination can be produced, is to let the inhibitory 
endbulb of L x in the following net impinge upon the output neuron 

of P of the net of Sect. 5* The number of the L's is ^+£-2. 



Figure 22 

(Drawn for t+L - 5.) 


We can also use this to fix nets constructed as in Sect. 5 
to represent a non-positive event by non—firing at j>+b, if we 
change the endbulb of Li which is to impinge on P to a set of 
excitatory endbulbs equal in number to the threshold of P, and 
also fire P itself at time 1. (If £+& ■ 2, no L's are added.) 

The devices of Figs. 21 and 22 seem artificial, and not 
likely to be found in organisms. We point them out to save the 
need for making an exception of non-prepositive events in the 
theory. If mechanical realizations of McCulloch—Pitts neurons 
are used in controlling robots, such devices might be useful. 
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The upshot of the analysis is that only by reference to 
artificially produced firing in inner neurons at t ■ 1 could 
an organism recognize complete absence of stimulation of a 
given duration, not preceded by stimulation; it would not know 
whether the stimulation had been absent, or whether it had itself 
meanwhile come into existence. 

If instead of the initially fired inner neuron of Fig. 21 
we use an input neuron subject to continual environmental stimu¬ 
lation, then all eventB can be taken to be prepositive by 
referring them to the class of input neurons as enlarged to 
include K. 

This is plausible biologically, If we also grant that 
the mathematical model is probably too exact in that it gives 
too much emphasis to a single neuron at a single moment of 
time (.0005 sec.). It is unlikely that any such input at a 
single moment would by itself result in any significant overt 
action or memory. 

Having chosen to Investigate a precise model, it is not 
to be expected that all aspects of this model will be equally 
pertinent to the reality from which the model is abstracted. 

6.4 Why consider indefinite events? Since the lifetime 
of an organism or machine is always finite, having an end as 
well as a beginning, why is it not sufficient to consider only 
definite events? 

The number of moments (identifying a moment with a synaptic 
delay of .0005 sec.) in a human lifetime of 100 years is of the 
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order of 3 x 10 12 . 

To construct a nerve net, treating events as definite, 
that would account for behavior at 60 years of age influenced 
by stimuli at 10 years, we would need chains of neurons of 
length 1.5 x 10 . If the event were at all complicated, we 

would need large numbers of such chains. Moreover, we would 
need further mechanism to provide for this same behavior occur*- 
ring at 6l years or 59 years due to stimuli at 10 years, or 
indeed for each value of d where d ranges from the smallest 
elapsed time after 10 years at which the behavior can be 
Influenced up to the greatest, and is measured in units of 
.0005 sec. We do not necessarily need a whole new set for 
each value of d, since many neurons can be made to serve in 
common for various values of d, e.g., the delay chains for 
various values of d greater than a given one dj could have their 
first dj neurons in common. But at least each Intermediate 
value of d would, up to the greatest In question, require some 
structural additions, new axons if not new neurons. 

All this would have to be duplicated for every sort of 
event which occurring at one time could influence behavior at 
all later times in life. 

18 

The total number of neurons is only of the order of 10 . 

To use definite events as a mathematical basis for explain¬ 
ing human behavior in all Its flexibility over a lifetime of 
3 x 10 12 moments thus appears altogether unrealistic. 

To emphasize what is meant, take the case of Solomon 
Grundy. On the afternoon of Monday he bums his hand on the 
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stove. Then one nerve net tells him not to touch the stove 
on Tuesday, a different one (at least In part) on Wednesday, 
and so on. 

If he outlives the life expectancy for which his delay 
chains are designed, he must thereafter suffer an advancing 
amnesia; for each day added beyond his expectancy at the end of 
life he completely forgets one day at the beginning. 

Humans and animals do not function in this way, though 
simple mechanisms for learning and subsequent forgetting in 
robots could be devised on this basis. 

Indeed, calculations on the amount of Information recorded 
in the memory (cf. McCulloch 19^9) make it difficult to explain 
memory entirely in terms of McCulloch—Pitts neurons on any 
basis, a fortiori , certainly not in such an uneconomical way as 
by setting up only nets for definite events. So it is neces¬ 
sary (if perhaps In the end it will not be sufficient) to go 
beyond the present stage of our analysis. 

It thus appears that the appropriate mathematical abstrac¬ 
tion for us now is to treat the problem of explaining behavior 
as though organisms and machines were immortal, having an 
Infinite future though a finite past. We want to provide for 
behavior that could be used ad infinitum . If merely the nerve 
net and effector mechanisms were immortal. 

By trying to provide for behavior over an infinity of 
time by a finite mechanism, we have a model for the real prob¬ 
lem of providing for complex behavior over a long finite 
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lifetime by a relatively small mechanism. 

The questions of reducibility of other mechanisms to 
McCulloch—Pitts nerve nets (not always without increasing the 
size of the mechanisms) is significant on this basis, but 
trivial on the basis of explaining behavior over a fixed finite 
time only. 

7. Regular Events ? 

7.1 " Regular events” defined ; We shall presently des¬ 
cribe a class of events which we will call "regular events." 

(We would welcome any suggestions as to a more descriptive term. ) 

We assume for the purpose that the events refer to the 
inputs up through time £ on a set of k input neurons Ni,,..,Nj c 

•mm 

the same for all events considered; but the definition 
applies equally well for any k > 1 or even for k * 0. 

The events can refer to the value of £. Our objective 
is to show that all and only regular events can be represented 
by nerve nets or finite automata. We have already seen in 
Sect. 6.5 why reference to the time is called for; but it may 
be illuminating to consider some examples from the point of 
view of solving given nets. 

Consider first the net of Pig. 22 taken by itself (the 
inhibitory endbulb from Li is superfluous now). The condition 
for Li's firing (under the assumption that it is fired Initially), 
i.e., the event represented by Li's firing, is given by formula 

Li(e) f E < 

ft 

McCulloch and Pitts use a term "prehensible, introduced 
rather differently; but since we did not understand theli 
definition, we are hesitant to adopt the term. 
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In defining "regular events " we shall build on the notion 
of a definite event of length as originally Introduced in 
Sect. 5.1 and completed by adding 2. £ > ^ to the definition in 
Sect. 6 . 3 * 

But now we extend the class of definite events by providing 

that the description of such an event may also contain the 

specification that the first moment is 1 ; i.e., that Srfri - 1 , 

i.e., that £ Events with thiB specification we call initial . 

k£ +1 

So now there are 2 s definite events of length £ with k 

k£ 

Input neurons, namely the 2 2 as before (non—initial) and 
the 2 a Initial ones. 

Now each event we shall build up, starting from (and 
k£ +1 

including) the 2 2 definite events for each will be Inter- 

pretable in the follow:ng way. The statement that the event has 
occurred (ending at time £) is equivalent to the statement that 
one of a non-empty finite or Infinite class of definite events 
has occurred (ending at time £). (More precisely, the class may 
be infinite if the value of £ is unknown, but for a fixed value 
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of £ there are, of course, only finitely many events which could 
have occurred.) 

Our olass of regular events will be defined inductively, 
starting with the definite events, and using three operations 
EVP, EF, E*F; i.e., it shall be the least class containing 
the definite events, and closed under these operations. 

Given any events E and F already built up from definite 
events by zero or more applications of the operations, by the 
event EVF we shall mean the event which occurs if E occurs or 
F occurs. In other words, the class of definite events which 
can constitute an occurrence of E V? is the sum of the respec¬ 
tive classes for E and F. 

Clearly the operation is associative; i.e., (EVF)VGg 
E V(fVG)j so the parentheses can be omitted. The reason for 
writing equivalence here with four bars will appear in Sect. 7.3* 

For example, if E and F are definite events of durations £ 
and m, respectively, then EVF is an event which occurs exactly 
when an event (of length £) belonging to E occurs, or one (of 
length m) belonging to F occurs, or both. One might be tempted 
to regard this as a definite event of duration max(^,jn); but 
this would be wrong, since supposing £ > m it could occur when 
£ < ^ « max(^,m), namely by F occurring. Also, this would not 
give what we want when either of the next two operations is 
applied. 

Given any events E and F already built up by the operations, 
by EF we Bhall mean the event which consists in E having Just 
occurred preceded immediately by F having occurred. Thus, EF 
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has occurred ending at time £, exactly when one of the 
class of events which can constitute an occurrence of E (say this 
one is of length £) has occurred in time Sri* 1 to £, and one 
of those which can constitute an occurrence of F (say this one 
is of length m) has occurred in the period £-£-m+-l to £— 

(Note: We have chosen our notation EF so that we proceed back 
into the past in reading from left to right.) This'Operation 
is also associative. 

For example, if E, F, and G are definite events of lengths 
m, and n, respectively, any occurrence of (EVFjG will be a 
definite event of one of the lengths ^+n or nH-n. (Say £ > m. 

By refraining above from interpreting EVF as of duration 
now when (EVF)G occurs by F occurring ending at time £, the 
preceding occurrence of G must end at time £-m, not ) 

If E is an Initial definite event, and F is any event of 
our class, then EF Is an event which never occurs; since for E 
to occur, its first moment must be t ■* 1; so under the interpre¬ 
tation in Sect. 6.5, EF i is impossible for any one of the 
definite events F 1 , F 2 ,... whose occurrence can constitute an 
occurrence of F. Thus, in this case EF is represented by the 
firing of P in the net of Fig. 9* 

If E and F are events already constructed, then by E # F 
we shall mean the event which consists of zero or more consecu¬ 
tive occurrences of E preceded by one of F. That is, E*F can 
occur whenever 

n times 

TTlsF 


occurs for some n £ 0. 
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The reason we do not define E* separately as a unary operation 
(expressing that E has occurred consecutively zero or more times) 
instead of E*F as a binary operation, is that then for n « 0 
an occurrence of E* would be of duration 0; but (at least for 
convenience) we are requiring the lengths of our definite events 
to be always > 1. 

To say that E has happened one or more times we can write 

E*E. 

For example, if E, F, and G are definite events of lengths 
m, and n, respectively, an occurrence of (EVF)*0 must be 
of a definite event of one of the lengths + bm + n 
(a, b£0). 

We reflect now that we have two systems of notation for 
events: (A) logical notations for definite events as used In 

Sect. 5 (with the addition of £ ^> in Sect. 6.3 and £ m ^ 
above) and for some other events in Sects. 6.1 ff.; (B) our 
newly introduced notations for regular events starting with 
single capital letters as representing definite events. 

There will be ambiguity if we use (A) as the starting 
point for (B) Instead of capital letters, unless we are careful 
to show the durations £ 1 >•••°f each of the definite events 
used as the units for the construction of the regular 

events. 

The question of translatability between the systems of 
notation (A) and (B) In either direction has not yet been 
examined thoroughly. 
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However, we have verified that the notations (B) can be 
translated into notations (A) with, of course, a sufficient 
amount of mathematical apparatus added to the logical notations. 

The details are technical and are given in Appendix 2. 

But it may be instructive now to give a few simple examples 
of translation in the other direction, i.e., from (A) to (B). 

The conventions regarding parentheses are those of algebra 
with E\/F, EF, and E*F analogous to e+f, e£, and e a f. 

Also, the associative law (E*F)G 2 E*(FG) permits omitting 
parentheses, as well as the two associative laws already mentioned. 

The event of duration which happens for all inputs over 
the interval £—^+1 to £ we call the identical event of length 
for ^ * 1 we write it as I, then in general 1^ (S I ... I to ^ 
factors). 

Let the result of adding £ = 1 to the specifications for 
a definite event E, to make it initial, be written E°. 

For any event E of length 1 the negation E is also definite 
of length 1. 

For the present illustrations, let the event of length 1 
that N fires at time £ (in symbols, N(£) be written simply N; 
that K fires simply K; and that both K and N fire be written L. 

Now the events described as follows in the left column are 
expressed by the corresponding notations in the right column. 


(See next page) 
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<«)»t < 

I*N 

(»)t < 

N*N° 

<!£)„ < E |(“) £(*)„ < t <; 

N*K 

'(&)„ i u> 4 < ^3 

N*L°V N*LK*K° 

N(j:) for at least two values of _t < £ 

I*NI*N 

N(_t) for exactly one value of t < £ 

^♦jPv^nTT*!? 0 , 
call this M 

N(t_) for an odd number of valueB of t < £ 

(TT*N!T*N)*M 

£ > 3 

I 3 

£ “ 1 

1° 

£ * 1 mod 3 

(I 3 )*I° 

£ < 5 

I°VlI 0 VI 2 I° 


7.2 An algebraic transformation : We list several equiva— 


lences: 




(1) 

(EVF)VG 

S E 

\/(FVG). 


(2) 

(EP)G 

— 

E(FG). ( 

Associative laws 

O) 

(E*F)G 

— 

E*(FG). 


(4) 

(E UF)G 

~ 

EG VFG. 


(5) 

E(F VG) 

lii 

EFVEG. \ 

Distributive laws 

(6) 

E*(F yG) 

= 

E*F V E*G. J 


(7) 

E*F 

— 

F VE*(EF). 


(8) 

E*F 

= 

F VE(E*F). 
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Under the definition Just given, each regular event is 
obtained by building it up from certain definite events as 
the units by zero or more applications of the three operations. 
Of course, these constructions are by no means unique. 

Lemma 1 . For any s > 2s Every regular event can be 
expressed as a finite disjunction of one or more regular events , 
each of which is either definite of length < s or is an event 
constructed out of units each of length ^ a. (Also true tri¬ 
vially for jb =» 1.) 

Of course, we can always understand there is at most one 
of the latter, since any disjunction of them is again one. 

We write out the proof for the case a *• 2. 

The lemma is true when the given event is definite; then 
there Is Just one term in the disjunction, which is of the 
first or the second kind according as its length is 1 or more. 

Likewise, if E and F each have the property described in 
the lemma, so does EvF. 

Now say E and F are as described, and consider EF. By 
use of the distributive laws (4) and (5), EF is equivalent to 
a disjunction of terms, each of which has one of the following 
forms 

E l F l , E^F 1 , E*F^, 

where in each case 1 indicates a definite term of length 1, and 
Indicates a term composed out of units each of length > 2. 

Now a term E l F l can be construed as a definite event of 
length 2; so it is of the second kind for the theorem upon con¬ 
sidering it as one of the units. 
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Also is of the second kind. 

Now consider E^F 1 . Using (2), ( 3 ), and (4), the F 1 can 
he moved progressively inward until finally F l occurs only in 
parts of the form HF 1 where H is definite and of length > 2. 

Then each such part can be taken as a unit, which will be of 
length 3 . 

For the last form E X F ^ 2 ^, we proceed similarly using (2) 

(from right to left), ( 5 ) and ( 8 ) (in combination with ( 5 ) and 

( 2 )). 

Now say E and F are as described in the theorem, and 
consider E*F. By use of ( 6 ) we can then get E*F equivalent to 
a disjunction of terms of the two forms E*F* and E*F^ 2 ^. For 
illustration (noting the remark just following the theorem), 
say, e.g., E is El VE& \/E^ a ^. So we have now two possibilities, 

(E{yEi VE^ a h # F x , (El V'e£v'e( , ))*F^. 

Consider the former. This is an event of which an occurrence 
must consist of one occurrence of F 1 followed by n > 0 occurrences 
of various of the events El, E£ and E^. Let G|,...,Q^ be all 
products of two of El, e£, and E^j i.e., Gi is EjEl, G 2 is 
EiEi, etc. Then an occurrence of the event is the same as an 
occurrence of one of F 1 , EjF 1 , EiF l or E^F 1 , followed by zero 
or more occurrences of any of Gi,...,G^. Thus, in symbols (and 
using (6) next and then (7))i 
(El V Ei V e(*))*F x 

s (Gi V* •. VG 9 )*(F‘ V ElF 1 V EjF 1 V E^F 1 ) 

= (Gx V. • • V / G 9 )*F l V (G 1 v - • • V / G 9 ) # (ElF 1 V' E£F x V E^F 1 ) 
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sp 1 v(Oi v/-.. vg 9 ) # (Oi v... vo 9 )p l v(a t v..o v'Qg)* 

(EjPVBiP 1 VE^^F 1 ). 

Now each of Gi,...,Gg can be handled as was one of E l F*, 
g(a)yi^ £*p(a) i n the case for EF above; then GiV 1 ... VG^ Is 
composed out of units of length ;> 2. Then by the method for 
E (a)pi i n the casc f 0r ef above, (Gi V... VGg)F l and E^F 1 are 
likewise, while EiF 1 and EiF* are definite of length 2. So the 
entire expression obtained last is of the desired form. Like 
arguments apply to (El VEi VE^ a ^)*F^*^. Finally, any disjunc¬ 
tion of expressions of the desired form is of the desired form, 

7.3 Identity and equivalence : In dealing with regular 
events, special care is necessary to distinguish between senses 
of "equivalence," As we Introduced them, any regular event 
is identified with a class Of definite events; and two regular 
have thus far been treated as equivalent only if these classes 
of definite events for the two are the same. 

An event Is a partition of all the possible inputs over 
the whole past for the nerve net into two classes, those inputs 
for which the event occurs, and those for which it does not 
occur. 

What we have called a "non—initial definite event of 
length is a partition of all the pasts for the net into two 
classes, such that all pasts of length < ^, i.e., for whioh 
£ < ^, are in the second class (those for which the event does 
not occur), while those pasts of length > ^ are in the first 
or the second class according as the Input over the last ^ 
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moments has or has not a certain property; i.e., the classi¬ 
fication is independent of the input prior to £-^+1* 

But could two non—initial definite events which are dis¬ 
tinct in the value of ^ or the property over £—^+1 to £ be the 
same as events? 

They could in one case, namely when the property over 
£-^+1 to £ is impossible of occurring; this is the case which 
was treated by Fig. 9 in Sect. 5-2. These definite events of 
length X for various values of ^ > 1 are all the same as events. 
We may call this event, which never occurs, the Improper (or 
impossible) event . 

Outside of this exception, an event can be a definite 
event in only one way. For suppose we have an example of an 
input over time 1 to £ for which the event occurs. Then we 
may seek the least X. £ such that the event also occurs when 
the value of p is changed to ^ and the input over time 1 to | 

(■ fi) is what it was formerly over time £-£+1 to g. This £ must 
be the length of the event; and the property of the input over 
the last moments which defined the event is obtained by eon— 
sidering what inputs over this time give occurrences of the event. 

Similar remarks apply to "initial definite events of 
length " Here all pasts for which £ / ^ are in the second 

class. The initial definite events of length £ which never 
occur are all the same as events. 

Combining the eases of non—initial and initial defi¬ 
nite events, an event can be a definite event in only 
one way (i.e., either non—Initial or Initial, but not 
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both, with only one length and with only one property of 

the input over £-^+1 to £), except for the improper event which 

can be construed as either non—initial or initial and with 

k? . 

any So actually there are only 2 a — 1 distinct definite 
events of length £ with k input neurons; in Sect. 7.1 we counted 
the improper event twice, once as a non-initial and once as 
an initial event. For k ® 0, there are thus Just 5 events of 
length £ » the possible non—initial one 1^, the possible initial 
one and the impossible one T. 

Now consider a regular event of the form EVF, where E 
and F are definite of length Quite evidently the E and F 
are not uniquely determined from the event. For example, there 
might be two k x £ tables exactly for which E occurs, a third 
for which F occurs. By recombining, taking E* as occurring when 
the first table applies, and Fi when the second or the third 
applies, we get the event as EWFi with different components, 
or indeed, the event can be considered as one event of length 

Now, in fact, our transformations of events in Sect. 7.2 
were suoh as to preserve the class of definite events under¬ 
lying a given regular event, except that sometimes, e.g., 

E^F 1 was reconstrued as a definite event of length 2. To make 
it exact what transformations shall be allowed, we can reconsider 
a regular event as given by saying which of various tables of 
length £ for various with or without specification that 
p - £ (rather than merely £ > £) would describe an occurrence 
of it. For if it is given by saying which of various definite 
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events would describe an occurrence of it, then we can replace 
each of these by the tables (zero or more up to 2 !£ of them) 
which constitute an occurrence of the respective definite event. 
We shall say that two regular events, as given in the notations 
of Sect. 7.1 starting with specified definite events, are 
Identical if the resulting classes of k x ^ tables (for various 
20 are the same. We write identity by HE. 

The empty class of k x ^ tables goes with the improper 
event; call this event T. We have: 

(9) Eyl m TVE = E. 

(10) ET s I. 

(11) E*T *5 T. 

(12) T*E S E. 

These permit simplifications of events into which T is built; in 
fact, all T's can be removed, unless the whole becomes X. 

Now, unfortunately, given an event as simply a partition 
of the possible inputs over the whole past for the net, the 
class of k tables in terms of which it can be constructed 
as a regular event is not unique. 

Consider the example of NyNI*I and N, where k « 1, and 
N signifies the event of length 1 consisting of the firing of 
the one input neuron N at time £. 

The only k x £ table for N is that having a 1 in its one 
position. But NVNI*I has this table, also both tables of 
length 2 agreeing in the first row, also all four tables of 
length 3 agreeing in the first row, etc. 
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But N V NI*I and N each occur. If and only if the Input 
neuron fires at time so as events they are the same. We 
call this sameness equivalence , and write N VN*I 2? N. 

The importance of the distinction is that from EsP we 
can infer EG S pa, GE i= GF, E*Q«F*G, and G*E S G*Fj but we 
cannot make the first and third inferences in terms of equiva¬ 
lence = . In particular, N \/NI*I = N, but not (N >/NI*I)N= NN. 

(Of course, E *§ F implies E 2 Fj but not conversely.) As another 
example, 1*1 = 1*1°, but I*IN ip I*I°N = I (by (l?) below and 
( 11 )). 

Summarizing, our theory of regular "events," with our 
operations EVF, EF and E*F and the relations apply to classes 
of k x jf tables (fixed k and varying £) in terms of which we can 
represent the events, rather than to the events in the simple 
sense. More particularly, it is the two operations EF and E*F 
for which the class of tables for E, rather than merely the 
resulting event E, must be known, because the lengths of the 
tables enter into the meaning of the operations. 

It would thus be more explicit to say that we are dealing 
with a theory of certain expressions for events ("representa¬ 
tions" would be a good word, if we were not using it already in 
another sense). 

We now extend our notion of "prepositive" to initial events, 
by saying that all except the improper one (which is at the same 
time non-initial and as such prepositive under the definition 
in Sect. 6 . 5 ) are not prepositive . Single k x ^ tables are 
special cases of definite events of length so the definition 
applies to them. 
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Now we say that a regular event as given by a class of 
k x ^ tables (fixed k and varying is prepositive, if all 
the tables of the class are prepositive. 

In Sect. 6.5 we saw that prepositiveness was necessary 
and sufficient that a nerve net with all inner neurons ini¬ 
tially quiet constructed to represent a non—initial definite 
event of length £ when £ £ £ should also represent it correctly 
(without "hallucinations”) when £ < 

The extension to initial definite events preserves this 
as a necessary and sufficient condition for representability 
with all inner neurons initially quiet; the necessity is clear 
by reasoning similar to that in Sect. 6 . 3 , and the sufficiency 
holds because there is no such prepositive event except the 
impossible one. 

Furthermore, now a sufficient condition that in represen¬ 
ting an event it be possible to take all inner neurons quiet 
Initially is that there be a way of expressing the event in 
terms of definite eventB and our three operations for which the 
corresponding class of tables (or of definite events) is pre¬ 
positive. This will be Included as part of the next theorem. 

To get a necessary condition, we introduce the idea of 
a minimal set of k x ^ tables (fixed k and varying 20 for an 
event. Start with any set of k x ^ tables for the event, and 
to each table consider the least segment of it ending at time £ 
for which all backward extensions describe occurrences of the 
event. Replace the table by thiB. Carrying out the process 
for eaoh table in the given set, we get a minimal set. 
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The minimal set so obtained is unique for a given event, 
as one gets the same minimal set by first extending each given 
table to an Initial table in all possible ways (which method 
gives the complete set of tables, which is unique for the event), 
and then minimizing this (by the above process which leads to 
a unique result), we get the same class of tables as in minimizing 
directly. 

Of course, the method of minimizing is not described 
"constructively,” and one question which arises at once is 
whether a constructive minimization process for a set of tables 
corresponding to a regular event as expressed in terms of defi¬ 
nite events and our three operations exists. 

Another question is whether the minimal set of tables must, 
for a regular event, necessarily be one which corresponds to 
an expression for the event in terms of definite events and 
the operations. (The complete table does, as will follow from 
the proof of Theorem 6 in Sect. 9 .) 

We do not go into these questions, which one would naturally 
investigate if the study is to be continued. 

However, we can now say that a necessary condition .that a 
regular event be representable by a net with all inner neurons 
initially quiet is that the minimal set of tables for it be pre¬ 
positive. 

Some algebraic simplifications are possible when initial 
definite events er.ter into an expression for a regular event. 

Say E° is an initial definite event. Then for any regular event F; 

(13) E°F 5 T. 

(14) E°*F 5 T. 
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Used along with (2) — (4) (and for simplification (9) — (12)), 
we come out with the result that no initial event need enter 
into an expression for a regular event, other than as an 
"earliest" event in the following sense. 

For a given expression for a regular event in terms 
of definite events, we define recursively as follows 
which occurrences of definite events in it are 
earliest. 

In a regular event given as simply a definite event, that 
definite event is earliest . 

The earliest events in E and the earliest events in F are 

the earliest in EVF. 

The earliest events in F are the earliest in EF and in E*F. 

7.4 Representablllty of regular events : 

Theorem 4 : To each regular event, a nerve net can be con¬ 
structed which, when started in a prescribed way, represents 
the event by firing a certain inner neuron at time £ + 2 , if and 
only if that event has occurred ending at time £ inclusive . If 
the given event is prepositive, the representation can be by 
a net started with all inner neurons quiet . 

Proof is based on Lemma 1 , Sect. 7.2, for £ «= 2. The 
Theorem is true for T, by Fig. 9, Sect. 5-2; and for other events 
by Sect. 7.3, we can exclude T as a unit. 

So first we give the proof for the case of an event 
(not T) constructed out of units (not T) each of length 2 or more 
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This we do by induction on the number n of occurrences 
of units in the expression for the event. 

In the induction we will arrange at each stage that the 
neuron which is to fire at time jd+ 2 will be (as in Sect. 5.2, 
since T is excluded) one of threshold 1 impinged upon by only 
excitatory endbulbs (i.e., it effectuates a disjunction opera¬ 
tion) with no axons feeding back into the net. 

If n « 1, then the event is a definite one E. We have 
three cases, (a) E is prepositive, hence not initial. The net 
is as given in Sect. 5*2, the reasoning that this net works 
being supplemented as in Sect. 6 . 3 . (b) E is not initial and 

not prepositive. We use the treatment given in Sect. 6.2 
employing Fig. 21. (c) E is initial. Then we uBe an inner 

neuron as follows, treated for the net construction of Sect. 5*2 
as though it were an additional input neuron required for the 
occurrence of the event to fire at time e-£ +1 - 


+ 



Figure 24 

This, of course, is simply a neuron whose condition for firing 
is £ = 1. 

Now if n > 1, the event under consideration is of one of 
the forms EVF, EF, and E*F where E and F are each constructed 
from < n units. 
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First, suppose the event is EV/F. Then by the hypothesis 
of the induction we can construct nets to represent E and to 
represent F, say with representing neurons P and Q, respec¬ 
tively, each with threshold 1 and only excitatory endbulbs 
impinging, and with no axons feeding back. To represent EVF 
we "identify" P and Qj i.e., we replace them by a single neuron 
—call it P—having all the endbulbs which separately impinged 
on P and on Q, and we similarly identify the input neuronB 
for the two nets, i.e., the axons which led from 
Ni,...,N, in the net for E, and those in the net for F now both 
lead from Nx,...,N^. The construction can be diagramed as 
follows, using heavy lines to represent a number of axons.. 



The heavy bundle of neurons leading to P from the left are 
those which would be required in the net for E separately; like¬ 
wise from the right in the net for F. The bundle from Ni toward 
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the left, those from N x in the net for E, to the right in the 
net for F, etc. The fact that the output neurons for the two 
given nets had no axons leading back in insures that they 
still operate independently of each other in this combination. 

Next consider an event EF. In the construction out of 
units which we are using for E, consider those occurrences of 
units in it which are earliest. The events we are considering 

refer to k input neurons Ni,...,N^. Now consider the event 

» “ 

E which is obtained from E by modifying each earliest unit to 

make it refer to One new neuron N^ +1 which is required to fire 
at the second moment of each such earliest unit. There is 
such a second moment in the period of the unit, by our assump¬ 
tion in connection with the use of Lemma 1 that each unit is 

1 

of length > 2. Also, the resulting event E is regular with the 

same number of occurrences of units, since this change in the 

earliest units only gives an event with the same structure in 

terms of it3 respective components by the operations EV'f, EF, 

and E*F. So by the hypothesis of the induction on n, we can 

represent this event E by a net. However, we simplify the 

construction by leaving out the neuron of Fig. 21 in the case 

of earliest events in E which come under Case (b) for definite 

events. (By remarks in Sect. 7.4, Case (c) can be excluded.) 

Now the net for EF is obtained by identifying N^ +1 in the 
» 

net for E with the output neuron Q of the net for F, and of 
course, identifying N X ,...,N^ as input neurons for the two nets. 

I 

The output neuron is that for E . The construction can be 
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diagramed thus: 



The event E is positive, requiring a firing of at its 

second moment. But can be fired only at a time later 

than 2, since in its role of output neuron for the net for F 
it fires at time jc>+2 (£ > 1) where £ is the last moment of an 
occurrence of F. No "hallucination" is possible as a result of 

t 

leaving out the neurons of Fig. 21 for the units in E which 
were not prepositive, as this necessity that N k+1 fire at the 
second moment, which must be > 2, prevents. (In fact, the 
arguments of Sect. 6.3 that "hallucinations' 1 can occur when an 
event is not prepositive do not apply now, since some inner 
neurons of the net for F will necessarily be firing at the 
first moment of these units of E .) These remarks (with the 
avoidance of the neuron of Fig. 21) are necessary to establish 
the last remark of the theorem. 

We have lastly the case for E*F. As in the preceding 
case we modify E to E . Then we combine the nets obtained by 
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the hypothesis of the induction (omitting Pig. 21 in treating 

• . 

earliest units in E ) as diagramed thus: 



Under the assumption of infinite past time (as in Sect. 6.1), 
the firing of P could, of course, be explained by E having 
occurred repeatedly ad Infinitum into the past. But here we 
are understanding that the whole net is started in a certain 
condition, which is either that all inner neurons are quiet or 
that some inner neuron (or neurons, if we prefer) in the net 
for F are fired, according as F (and therefore E*F) is preposi¬ 
tive or not. Then as in the reasoning under the treatment of 

i 

EF, the net for E can only be a cause of P and N k+1 ‘s firing 
if there has originally been a firing derived from the net for 
F, which serves as an input into that for E that must be fired 
at the second moment for the latter. Of course, thereafter P 
will be fired on each repetition of E. (P and N k+ ^ must be 
separate, to meet the condition that the output neuron not 
feed back into the net.) 
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This completes the treatment for regular events construc¬ 
ted from units each of length > 2 . 

Now by Lemma 1, and using the method already indicated 
for the basis (i.e., for n « 1 ) to treat definite events of 
length 1 , and the method already used under the case of the 
induction step (n > 1 ) for EV'F to combine nets for different 
disjunctive members, we get the theorem for regular events in 
general. 


7.5 Discussion of the proof and further problems : As 
we have already remarked in Sect. 6.2, the use of a net with 
initially fired inner neurons seems unnatural. But this is 
unavoidable, if we are to represent non-preposltive events, 
since we mu 3 t (by examples such as are given at the beginning 
of Sect. 7.1) make our mathematical theory complete. A way 
of avoiding the use of such nets biologically, namely by con¬ 
sidering only events dated from some environmental stimuli, 
has been indicated (Sect. 6.2). 

A second respect in which the present proof seems arti¬ 
ficial and leads to complicated nets is in the use of Lemma 1, 
the proof of which Involves rather extensive reformulation of 
the events. 

If we deal only with events which are already expressed 
in terms of units each of length > 2 , or have the form for 
Lemma 1, the proof of the theorem is straightforward and the 
nets constructed are simple, i.e., of a degree of complexity 
corresponding very well to the complexity of the given descrip¬ 
tion of the event to be represented. 
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The necessity of using Lemma 1, if we do not restrict 
the events to be already expressed in the form for that lemma, 
arises from the fact that in Sect. 5 we needed a lag of 2 to 
represent definite events in general. 

Some simple events can be represented with lag of 1; and 
for these the units into which we feed the outputs from the 
nets representing the preceding events would not need to be 
of length > 2. Thus, to give a uniform treatment in proving 
the theorem, we resorted to a device (the proof of Lemma 1) 
or a restriction (that the property of Lemma 1 is already present) 
which can be dispensed with in many particular cases. This is 
why, e.g., in Sect. 6.2., simpler nets are available for repre¬ 
senting certain indefinite events than would be given by the 
method of proof of the present theorem. 

As was noted in Sect. often definite events can be 

represented by simpler nets by using a lag greater than 2. Then 
the method of net construction for the proof of Theorem 4 would 
require the use of Lemma 1 for £ » the greatest lag used in 
any of the units (or possibly not this great, depending on how 
the units enter). As the proof of Theorem 4 is given, also the 
net would have to be chosen so that the representing neuron 
appears as in Fig. 2j i.e., performs a disjunction operation. 

This could always be arranged by an increase of 1 in the lag. 

But also probably the proof can be adapted to apply directly, 
somewhat as the proof of Theorem 1 was generalized to get 
Theorem 2 (but we have not examined this in detail). 
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The difficulty which calls for Lemma 1 arises when we 
try to represent a repetition of some event which is shorter 
than the time necessary for the net to organize a representa¬ 
tion of It by the firing of a single neurot$ the solution by 
Lemma 1 consists in considering grosser events before attemp¬ 
ting to represent repetitions of them. 

If we consider that one or two synaptic delays are pro¬ 
bably not significant for determining behavior in an organism 
(as we remarked at the end of Sect. 6 . 3 }, it seems that the 
complication is connected with an over—refinement in our model 
of the biological reality. 

So we can urge that the methods of net construction used 
in proving the theorem are simple enough, granting that from 
the general method we can often start out to find simpler nets 
in special cases. 

The question may occur to the reader, why did we select 
the particular three operations E VP, EF, and E*F? When we 
say that the net constructions are simple, we mean simple for 
events already described from definite events by use of these 
operations. 

A pressing problem now is to consider what kinds of events, 
described originally in other terms, can be described in these 
terms; and so eventually what kind of behavior can be explained 
on the basis of nerve net control. 

This Is a problem one would naturally investigate in detail 
next. We have not done so thus far, since this report is 
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intended only to reduce to writing the author's thinking on 
the subject during August 1951* and not to try to carry the 
investigation further, except for the minimum amount of filling 
in details which was unavoidable in the process of writing. 

However, it is very plausible that the notation for 
regular events in terms of definite events combined by the 
three operations will prove handy in describing events. The 
simple examples given at the end of Sect. 7*1, and some others 
slightly more complicated, encourage this hope. 

On the other hand, given a description of an event in teims 
of definite events and the three operations, it will in some 
cases be difficult to see what the event consists of; we know 
of cases in which a very complicated description is actually 
equivalent to a much simpler one. (This, in fact, is usually 
the case for descriptions provided by the method of proof of 
Theorem 6 in Part II.) 

So there are problems of translatabillty in both directions 
between the notations for regular evente and other notations for ! 
events or descriptions of events in ordinary language. These 
problems have so far been touched only superficially, and are 
crucial for determining how far the present results carry us 
toward practical general techniques for construction of nets 
for given purposes. 

These questions are related to questions about transforma¬ 
tions between different expressions for the same event in terms 
of our operations. Can we obtain any normal forms, i.e., 
simplest forms or convenient standard forms, for descriptions 
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of regular events, to which given forms are equivalent in the 
sense of Sect. 7.3? Is there any decision procedure for the 
equivalence of two expressions for events (in the technical 
sense of modern logic)? These questions are closely related 
to questions raised in Sect. 7.3* 

Similar questions apply to identity in the sense of 
Sect. 7.3; but equivalence is the important relation for the 
applications of the theory. 

These questions are partly algebraic in character. Some 
questions are also raised in Part II and Appendix 2. 

Success in reducing, to terms of definite events and 
E\/F, EF, and E*F, events as expressed In ordinary language 
or as they arise in explaining organic behavior or creating 
robots for prescribed purposes would, of course, give a Justi¬ 
fication for our selection of the operations. 

Our actual reason for selecting them is that (as was men¬ 
tioned in Sect. 7.1) a converse of Theorem 4 will be proved 
in Part II. 

Thus, every event which can be represented must be expres¬ 
sible in terms of E\/F, EF, and E*F, starting from definite events. 

In particular, we have thus demonstrated that McCulloch— 

Pitts neurons can govern any kind of behavior which any other 
kind of digital automaton at-all can govern. This, of course, 
includes a number of special results which they obtained for 
alternative kinds of nerve nets, but is more general. 
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Having been first led to the three operations in connec¬ 
tion with the converse of Theorem 4 (i.e., Theorem 6 in Part II), 
it was natural to see whether the present theorem would hold. 

But, of course, the fact that our three operations are 
completely general (by Theorem 6) does not settle the question 
whether they will prove to be a convenient and practical way 
to deal with events. Possibly some other selection will prove 
to be more convenient. Or, we may add other operations and 
express these in turn in terms of our three. 

7.6 Conjunction and negation : We did not include the 
operations ^ (and) and (not) in our definition of regular 
events, because in the converse theorem (Theorem 6) we do not 
need to. 

In this section we will show that net constructions can 
be managed so that the two operations can be included. However, 
we will only treat them when applied to events already repre¬ 
sented by nets, and we will not thereafter use EF and E*F. 

From the converse theorem It will follow that any events 
we thus express using also £ and must be expressible without 
them. But the definitions obtained in this way are very com¬ 
plicated, and simple definitions do not appear to be immediately 
forthcoming. (But we have not examined the problem thoroughly.) 

We are not attempting to use ^ and Inside EF and E*F 
(except In the original constructions of definite events as 
the units) since we have not set up a representation of these 
operations in terns of classes of definite events or of k x ^ 
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tables. It does not seem to be immediate what is the best 
way to do this. 

Theorem 5: Each event construotlble from regular events 
fey-age oX the ... oper ati o n s & , V , of. the.-£EgpgplUonftl 

&aloulus Is representable with lag 2; i,e»^ a nerve, net... and_a 
neuron can be found, together with an initial state of the 
aet > so. that the neuron fires at time v+2, IX an d, only H L .the. 
ey.e nt . h as_ -t.a^.en_p.lace ending, at time p. 

Proof : Say the event is constructed by the operations of 

the propositional calculus from certain expressions for regular 
events. Consider any one of the latter. Wherever a part occurs 
in it of the form E*F, replace this by PV'E(E*P) using (8). 

After this, apply (4) and (5) whenever possible. Using also 
(2) if necessary, we are thus led to an expression for the 
original event by operations of the propositional calculus in 
terms of regular parts of the form Ei(...E n ) where Ei is defi¬ 
nite; for this purpose we take the V's which have been brought 
outermost in the expressions for the regular events as part of 
the construction in terms of the operations of the propositional 
calculus. Say there are m such parts; call their first factors 

Ej^( i » 1,... ,m), and the whole expressions e{— 

Let Et be E t ^' or Ei^—^ according as is > 1 or *» 1, 

where 1 has the meaning given it in the proof of Theorem 4. Now 
we can take exactly the same combination by operations of the 
propositional calculus of Ei v * /n , ... , E* 1 


- •R.(ill)" that the given 


event is of ), ... , eW(...eW ). This can 

2.(1) 1 "(m) 
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be treated as a definite event of length equal to the greatest 
length of any of its components, and a net can be constructed 
for it by Theorem 1, with input neurons Ni,...,N k and for 
each .1 for which n^ > 1 a neuron N k+ ^ required to fire at 
time g—for the event to occur. Feeding the outputs 

from the nets for E1—^ ... appropriately into thiB, instead 


of as before into respective nets for e(— 
the event in question. 


we get a net for 


PART II — FINITE AUTOMATA : 

8. The Concept of a Finite Automaton : 

8.1 Cell3 i Time shall consist (as in Sect. 3 ff•) of a 
succession of discrete moments numbered as integers. We shall 
mainly be concerned with the case of only positive integers, as 
in Sect. 6.2 ff, but will consider the case of all the Integers 
In Appendix 1. 

We shall consider automata constructed of a finite number 
of parts, each being capable of a finite number > 2 of states 
at any given moment. Call these parts cells . 

We shall distinguish two kinds of cells, input cells and 
inner cellB . Say there are k input cells and m Inner cells. 

An input cell admits 2 states 0 and 1 (or "quiet* and 
"firing"), which we consider to be determined by the environment. 

This restriction to 2 states for input cells is to make 
the notion of an input to the automaton coincide with the notion 
of Input to a nerve net as formulated in Sect. 4. But the 
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present theory would work equally well with more than 2 states. 
Nothing is gained thereby, however, as £ cells each admitting 2 
states could be used to replace one cell admitting any number £ 
(2 < 3 < 2 ^) states 0 , 1 , ... , £— 1 , where if £ < 2 & we could 
consider only inputB in which states ... , 2^-1 do not occur 
or identify all of these states with the state g —1 in all the 
operations of the automaton. 

The number of states of an inner cell is not restricted 
to 2 , and different inner cells may have different numbers of 
states. 

Say the input cells are Ni,...,N^ (k ^ 0); and the inner 
cells are (m > 1 ), with respective numbers of states 

81 ,...,^ (each > 2 ). 

The state of each Inner cell at any time t is determined 
by the states of all the oells at time t-1. Of course, it 
may happen that we do not need to know the states of all the 
cells at time b~l to infer the state of a given inner cell at 
time t. Our formulation merely leaves it unspecified what 
kind of a law of determination we use, except to say that 
nothing else than the states of the cells at b~l can matter. 

If time is given as beginning with t «* 1 , the state of the 
inner cells at that time is to be specified. 

A particular example of a finite automaton is a MoCulloch— 
Pitts nerve net. Here all the cells have Just 2 states, and 
the principles stated in Sect. 5, together with the arrangement 
of axons and the kind of endbulbs on each case, give the law 
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determining the state of each inner neuron at time £ from the 
states of all the neurons (or, in fact, from only those having 
endbulbs synapsing on the given one) at time t—1. 

Another example is obtained by considering neurons with 
"alterable synapses" or "alterable endbulbs" of the following 
kind. Each neuron may have besides the usual endbulbs also 
exeitatory ones whioh are not effective unless at some previous 
time the neuron having the endbulb and the neuron to which the 
endbulb is adjacent were simultaneously fired. If a neuron 

nil 

has r such alterable endbulbs, it is oapable of 2— states, 
according as it is quiet or firing and according to which of 
the r alterable endbulbs have thus far been made effective. 

Many other possibilities suggest themselves. 

8.2 State : With input cells Ni,...,Nj c and inner cells 
with respective numbers of states there 

If 

are possible at a given moment exactly 2— • s_i*...*s ra states of 
the entire automaton. We can consider each as a combination 

Jr 

of an external state, of which there are 2— possible, and an 
internal state of which there are £i*»***s fll possible. 

The law by which the states of the inner cells at time t 
are determined by the states of all the cells at time t—1 can 
be given by specifying to each of the complete states at time 
t—1 which one of the inner states at time shall succeed it. 

Now, indeed, there is no reason for our general theory 
why we cannot consider the entire aggregate of internal cells 
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as replaced by a single one admitting states. This 

normalization of our concept of a finite automaton is always 
possible, though we did not start out with it, because we 
were interested in making clear the application to such automata 
as a McCulloch—Pitts nerve net, where the cells are given cer¬ 
tain simple properties and are connected in a certain way. 

We could also restrict ourselves to one input cell, by 
scheduling the inputs on the k original input cells to come in 
successively in some order on the new one, which would alter 
the time scale so that k moments of the new time scale correspond 
to 1 of the original. Events referring to the new time scale 
could then be interpreted in terms of the original. However, 
we do not find any advantage in this reduction to one input 
neuron| so we do not use it. 

We will now assume that time starts with t « 1. Say we 

lr 

call the states a»,...,a r where r * 2—* s_i and the 

internal states b*,...,b where a * s_i We specify that 

Jib —- 

the internal state at time t ■ 1 be bp 

Under this assumption, the state at time t - £ is a function 
of the input over time 1 ,...,£ (including the value of £, or only 
this when k » 0). (Had we not specified the Initial state as bi 
the state at time £ would be a function of the initial state also.) 

So each of the states ai,...,a r corresponds to (or repre¬ 
sents) an event, which occurs ending at time £, if and only if 
the input over the time 1 ,...,£ is one which results in that 
one of ai,.*.,a being the state at time £. Thus, the automaton 
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can know about the past (inclusive of the present) only that 
it falls into one of r mutually exclusive classes (possibly 
some of them empty). 

Similar remarks apply to representations of the past by 
an internal state assumed at time £+-1# or by a property of the 
state at time £, or of the internal state at time £+1. For 
to say the internal state at time £+1 i3 b^ means that the com¬ 
plete state at time £ was one of certain ones, i.e., those 
which are succeeded by bj^ under the law determining internal 
state. So then the past falls into a class of possible pasts 
constituting the set sum of the classes represented by those 
complete states at time £, or in logical terms the disjunction* 
similarly, for properties of the state (similarly also, 
e.g., a property of the internal state at time £+£ for s > 1, 
whenever this property does not depend on the input over time 
£+1 ,...,£+£— 1 ). 

What sorts of events can be represented? The question is 
answered by the following theorem, referring, of course, to 
automata started in state bi. Had we not specified the initial 
state, we would merely add (or disjoin) the classes corresponding 
thus to the £ internal states, each in turn as initial state. 

Had we not specified past time to be finite, the state at 
a given time £ would not necessarily be determined by the input. 
The facts in this case (already mentioned in Sect. 6.1 for 
McCulloch—Pitts nerve nets) are dealt with in Appendix 1. 

As the concept of input is the same as in Part I, we can 
use the notion of "regular event” which was Introduced in Sect. 7. 
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9. Regularity of Representable Events ; 

Theorem 6: Por any finite automaton (in particular, for 
a McCulloch—Pltte nerve net) started at a given time t » 1 
with Internal state bi at that time, the event represented 
by a given state existing at time £ Is regular ; jL.e., the 
automaton assumes that state at time £, If and only If a cer¬ 
tain regular event occurs ending at time £. 

Proof: Since the Initial internal state is specified, 
there are 2— initial states (the results of combining the 

Ir 

given initial internal state with each of the 2— possible 
external states at time t ■ l) from which the automaton could 
start at time t ■ 1 to reach the state in question at time 

1 - £• 

So if we can show that the automaton can start from a given 
state at time 1 and reach a given state at time £, if and only 
if a certain regular event occurs ending at time £, then the 
theorem will follow by taking the disjunction of 2— respective 
regular events, which (by Sect. 7.1) is itself a regular event. 

Given any state a at time t—1 (t—1 £ 1), exactly 2— states 
are possible at time Jt, since the internal part of the state at 
time t is determined by a, and the external part can happen in 

lr 

2 — ways. 

So we have a one-many relationship between states. Now 
invert this relation and consider for any state a at time it 
what states at time t—1 are compatible with it (there may be 
none, one, or more than one)j say a is in relation R to each 


of these. 
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The next part of our analysis will apply to any binary 
relation R defined on a given set of r objects (called 

"states"), whether or not it arises in the manner Just des¬ 
cribed. 

Consider any two a and a of the states, not necessarily 
distinct. We will seek a characterization of the strings of 
states d^d^ji* • *d^ for which d^ is a, d 1 is a, and for each 


i (l_ * 1,...,^—1) d 1+1 is in the relation R to d^j call these 
strings which connect a to a. 

Let A be a class of such strings. We call A regular , if 
A can be described by an expression built out of the following 
operations (chosen in analogy to the definition of regular 
events in Sect. 7.1.) 

The empty set and the unit set consisting of Just aj^ for 

any i_ are regular . If A and B are regular, so is their sum 

which we write AVB. If A and B are regular, so is the set, 

written AB, of strings obtained by writing a string belonging 

to A just left of a string belonging to B. If A and B are 

n factors 

regular, so is A*B which abbreviates A • •'* AB (n > 0), 
i.e., the siim of these classes for all n ^ 0. 

Now we prove the lemma by induction on r that the strings 
dy. . .d-^ connecting a to a form a regular class. 


Basis ; r » 1. Then, of course, a is a. If a R a (i.e., 
if not a R a), the olass of the connecting strings is simply 
the unit set consisting of a (as string of length 1), which is 
regular. If a R a, then the class is ja, aa, aaa, ... j , which 
is regular, since it can be written A*A where A ■ fa 1 . 
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Induction step: r > 1. 

Case 1 : a « a. In this case any string leading from 

a to a is of the form 

a ^ a ..^ • • • a .... a^ 

no a's no a's no a’s 


Figure 28 


where each arrow is either empty (this being possible only if 
a R a), or stands for a string without a in it. 

Let ex,..., e be the states e such that a R e, but e / a. 


and fx,...,f h the states f Bueh that f R a but f / a. Now any 
string of the kind represented by the arrow (when the arrow 
does not represent the absence' of a string) must start with 
one of ex,...,e and end with one of fi,...,f h . For each pair 

e i f J/ by bhe h yP° th « sl 8 of the Induction, the class of the 


strings leading from e 1 to f^ (without a in it) is regular. 

Say Bi,...,Bg^ are these regular classes; let A be ja|. Now 

if a R a and the B's are not all empty (Subcase i), the class 
of possible strings a-»for Fig. 28 is AV A(Bi V ... VB^); if 

a R a but all B’s are empty (Subcase ii), it is A; if a R a and 
the B's are not all empty (Subcase iii), it is A(B X V ... V B^); 


and if a R a and the B's are all empty (Subcase iv), it is 
empty. In the first three subcases, let C denote the class mai — 
tioned, which is non-empty and regular. Now in these subcases, 
the class of Btrings leading from a to a (as in Fig. 28) is 
C*A, while in the fourth subcase it is simply A. 
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a ^ a. Now we have instead of Fig. 28 the 


a ————— ^ ... a .. - . .a . . — 

no a's no a's no a 1 s or a’s 

a —————————^ ... a .— ■ ^ a 

no a’s or a *b no a's or a’s 

Figure 29 

The treatment is similar. For example, in the oase the classes 

of strings represented by "a-by "a-" in 

no a's no a'8 or a's 

the middle, and by "a-" at the right, are none 

no a's or a's 

of them empty, call them C, D, and E, respectively, the class 
of strings for Fig. 29 is C*ADE*X, where JT - jaj. 

So the lemma is proved. Now we return to the point where 
we were in the proof of the theorem. We wish to show that for 

lr _ 

given state a at time £ and each of 2— possible states a at 
time 1, that a holds at t «* £ and a at t « 1, if and only If 
a certain proper event (different in general for each of the 
2— IT's) occupies the time 1 to £. 

Now by the lemma, the strings of states which can lead 
from a to a form a regular class. If that class is empty, 
then the event is the improper one, which Is regular. If that 
class is not empty, consider the expression for the class as 
a regular class of strings. We build a corresponding expression 
for the event as a regular event by translating each unit 
class A. » fa.l (for each state a^) into the definite event E^ 


Case 2 : 

following: 

a- 

no a's 
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of length 1 which occurs at time £ exactly If the Input at 
time £ is the external part of the state a^,. (If k - 0, E^ 

is the event I of length 1 which always occurs# having no other 
features. However, it may be initial 1, that is, E°. The only 

other event of length 1 in this case is the improper one T.) 
Having done this, then the operations EN/P# EF, and E*F for 
building regular events parallel those AVB, AB, and A*B for 
building regular classes of strings. The earliest units 
(Sect. 7.3) in the expressions obtained should be marked as 
being initial. 

This proves the theorem. No attempt has been made to 
consider, in this proof, how simply the event represented by 
the state at time £ can be constructed as a regular event. We 
have worked out some simple illustrations in which very compli¬ 
cated expressions stand for regular events capable of simple 
ones. The expressions obtained have entirely initial events 
as earliest units, and are built of units of length 1. It is 
clear that in most examples great simplifications can be obtained 
by use of equivalences (Sect. 7.3); but no study has yet been 
made of the possibilities for proceeding systematically with 
such simplifications, or of rearranging the proof of Theorem 6 
to come out directly with simpler expressions when possible. 

The study of the structure of a set of objects a*,...,a 
under a relation R, which is at the heart of the above proof, 
might profitably draw on some algebraic theory, since it is 
possible (though whether profitable or not we do not know) to 
see the situation as a generalization of permutation groups. 
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Corollary : The event ending at time £ represented by 
each of the following Is likewise regular : (a) a certain, 
internal state at time jh- 1. (b) a property of the state at 

time £. (c) a property of the Internal state at time |H*1. 

(d) a certain internal state at time £4-s for a given s > 1, 
when this does not depend on the Input over £4-1 to jh-s—1. 

(e) a property of the internal state at time jw-s for a given 
3 > 1 , when this does not depend on the input over £+1 to 
£+ 8—1 . 

Proof : As remarked at the end of Sect. 8, each of these 
is equivalent to one of certain states existing at time £j so 
the event represented is a disjunction of the regular events 
given by the theorem for the latter, and hence is regular. 

This corollary brings our result now Into correspondence 
(as converse) of the result in Sect. 7. There we represented 
an event by firing a certain neuron at time jh- 2. This is a 
property of the internal state at time p+2, sinoe it means the 
internal state then is one of different ones (according 

to the states of the other m—1 inner neurons). 

Incidentally, the representations In Sect. 7 by firing a 
certain neuron at time £+2 are equivalent to representations 
by a property of the state at time £, namely by the property 
which those states at time £ share which will lead to the firing 
of this neuron at time £f2. 

The event which is represented by a state a Is the solu¬ 
tion in the sense in which McCulloch-Pitts speak of "solution” 
for the case of nerve nets, except that we give the solution 
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for a given internal initial state. A solution without pre¬ 
supposing an initial internal state would then be obtained as 
a disjunction of the solutions for us for each of the 5.i*****B m 
(for nerve nets, 2—) internal states. 

Appendix 5 contains an example of an event which cannot 
be represented in a finite automaton. 

It is, of course, essential for our arguments here that 
the number of cells or parts (under our first definition of 
a finite automaton) and the number of states for each, be 
finite, so that the number of complete states is fixed in 
advance. A machine of Turing (1937) which is supplied with an 
unlimited amount of tape, is not a finite automaton in our 
present sense, since, although In its operation only a finite 
number of squares of tape are printed upon at any time, there 
is no preassigned bound to this number. 

The Turing machine could be thought of as a finite auto¬ 
maton, which is also able to store information in the environ¬ 
ment and reach for it later, so that in certain cases the inputs 
are identified with inputs at earlier times or with states of 
certain inner cells at earlier times, and thus the present input 
is not entirely independent of the past. Whether this compari¬ 
son may lead to any useful insights Into Turing machines, or 
reciprocally into finite automata, remains undetermined. 



RM—704 
Page 87 


APPENDIX 1 : DEFINITENESS OF EVENTS REPRESENTABLE IN A FINITE 
AUTOMATON WITH AN INFINITE PAST: 

Theorem 7 : Every event E ending at time £ representable 
by a certain state existing at time £ ( or by one of the other 
methods listed In Corollary Theorem 6) In a finite automaton 
with an Infinite past Is definite . 

The result was cited In Sect. 6.1. The notion of auto¬ 
maton to which we refer Is given In Sect. 8. 

Proof : With k input cells, the complete past Is generated 

if 

by choosing between a finite number of 2— possible inputs at 

If 

time t » £, then between a finite number 2— at time t » £—1, 
etc., ad Infinitum . 

By a theorem given by Brouwer (1924) and alBO by Konig 
(1927), if for each infinite past (l.e., for any such choice 
sequence) it is determined at some finite stage whether the 
event occurs (ending at time £) or not, then there must be a 
number N > 0 such that, whether the event occurs or not is known 
for all pasts (l.e., all choice sequences) from only that part 
of the past occupying the time £-N to £. In this case the event 
would be definite of length N+l. (Brouwer’s proof of the theorem 
is intended for readers acquainted with the Intultionistic set 
theory, and the main effort in his proof is to demonstrate the 
theorem intuitionistically.) 

Now we show that indefinite events are not representable. 
Contraposing the mentioned theorem, we conclude that for an 
indefinite event, there is some particular infinite past such 
that for every u it is not known from the part from £-u to £ 
of the past whether the event occurs or not. 
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Case 1 : The event E does not occur for this particular 
past. Then for every u there Is a past coinciding with the 
given one over the time g-u to £ and diverging from it prior 
to £--u, for which the event occurs. 

Now suppose the representation of the event Is by a pro¬ 
perty of the state at time p; say the notation is arranged so 

that the states which have this property are a*,...,a and 

* 1 

the states which do not are a r ,. .. ,a r . 

Now consider the set S of all the sequences of states 
d O d l d 2 * * * com P atlb l e with the present state being one of 

a ri+l'* * * ,a r ; 1,e, » d Q ls one of a n+l’’ * * ,a r* and each bas 
as its internal part that which is determined by d 1+1 There 

are r-ri choices for «3 q, at most r for d t , at most r for d«, etc. 

But for any u there is a past coinciding with the given 
one over the time £-u to £, and diverging from it before that, 
along which the event does not occur. Along this past, any 
sequence d^d^dg ... must belong to S and must in its first 

u+1 choices d Q d 1 ...d u be compatible with the given past (as 

selected above); i.e., the external parts of d^, ...,d u 

must be the Inputs over the last u+1 moments of that past. 

Now by Brouwer's theorem (contraposed) there must be an 
infinite sequence rigd^dg... in the set S which is compatible 

with the entire given past (along which E occurs, but from 
every finite segment of which a past diverges along which E 
does not occur). 
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But dg is one of the states a ri +i>**‘* a r , contrary to 

our assumption that E is represented by the state at t « £ being 
one of . Thus, E cannot be so represented. 

If we had assumed simply that whenever E occurs, the 
present state must be one of a l# ...,a , the above considera¬ 

tions show that there must also be examples in which the present 
state will be one of aa without E having occurred. 

A, 1 

Case 2 : The event E does not occur for this particular 
past. The reasoning already applied gives the absurdity of E 
being represented, hence of E Itself being represented, by a 
property of the state at t ■ £. 



RM—704 
Page 90 


APPENDIX 2 : PRIMITIVE RECURSIVENESS OF REGULAR EVENTS : 

Theorem 8: Every regular event la primitive recursive . 

The terminology in the theorem is from the theory of 
recursive functions and predicates as developed in the last 25 
years. A book by Peter summarizes the theory, also a book by 
the author which it is hoped will soon be in print. 

The formulas given below "place" the regular events in 
relation to number—theoretic predicates studied in the theory 
of recursive functions. Although we have not pursued the mat¬ 
ter further than to get one way of expressing regular events 
recursively, possibly useful characterizations may be obtain¬ 
able in this direction. 

We already know from Sects. 5, 6.5, and 7.1 that number- 
theoretic formulas can be constructed to stand for definite 
events. The symbolism required can be seen by Inspection of 
the examples given. Terms g—1, £-2, ... are used only when 
they are greater than 0, as is insured by adding £ > ^ 

(Sect. 6.3) or now sometimes £ » ^ (Sect. 7.1) to the expres¬ 
sions as given in Sect. 5. 

The range of the variables in the theory of recursive 
functions is customarily 0, 1, 2, ... rather than 1, 2, 3, ... . 
To avoid having to reconstruct the notations in that theory 
for the present application, let us in this appendix suppose 
the time scale for events starts with _t ■ 0 instead of t ■ 1. 
Slight changes are then required in the formulas for definite 
events. Incidentally, now £—1, £-2, ... are used in the sense 
of £*-l, j>*-2 in the theory of recursive functions. 
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We shall simultaneously build expressions for regular 
events and for the lengths of the definite events of the class 
of definite events which we use in characterizing the regular 
events. More precisely, we describe to each regular event E 
expressions A( E ), A{£,n), 6(2.)* and ^/<(n) such that 

has occurred ending at time e| H (®2 .)n<f ( e )A(£»!1 )=: A(£), 

where for each n,<6(£)* A(£,n) describes a definite event of 
length y^(n). Here£(£) is the number l) of definite events, 
in the occurrence of one of which the regular event consists. 
{0ur6(|>) is not necessarily the least number of such definite 
events, but is the number we use in our construction. Also, 
the £ (g) regular events need not all be different.) 

For a definite event used as a unit in the construction of 
a regular event, £(£) « 1 (so n has only one value 0); 
and A( E ) is the expression already mentioned for the event; 

/\(£,n) can be simply A{ij») , or A(p) % n « 0 if we wish n to appear 
explicitly in this case; while_y k (o) ® £ where £ Is the length 
of the definite event under consideration. 

For a regular event of the form Fv'G, say that B(g,n), 

% (£) and y'in) are the "A(|>.,n) rt , ”£(£)" and V^n)" for F, 

and C(g,n), ^(g) and ^(n) are those for G. 

Now take 

A(p) 3 (En)^ <6(£) A(jp,n) 

* 

= (^)n<6(£) {?< ^E) & 5(£»njj V [n> ^( E ) & C(£,ni jf < £ ))} 

(the scope of the prefix (!E.) n < £ (£) ln the last being A(£,n)), 
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where 


f >An) if n < % (p ), 

6(E) ‘ j|^^ E ),in>/( E ). 

Let a/bj = the quotient, andy^(a,b) * the remainder, 
when an integer a is divided by a positive Integer b. Note 
that as a ranges from 0 to kb—1 (b / 0), the pair of quanti¬ 
ties a/bj , ^(a^b) range over all pairs of numbers x,jr with 
x < k, £ < b. 

Now for FG, given expressions for F and for G as before 
by the hypothesis of the induction, we have 

A(£) = (En) n< e(£) A( £ ,n) 

s (5iL>n<e(E) [1 tc £(E -^[E/n ( E)J )* 

/*(n, 7f (r))) j , 

where £ (£) - (r) 7[ (£) 

>*(n) - >/( [n/7l(£)] ) + ? if (h,Y|( £ ))). 

For the remaining case we use the function (a^ defined 

thus. First let £^ - £the i—1th prime counting 2 as the O—th^ . 

(So £q ■ 2, pi * 3, £5 ■ 13, etc.) Now 

/ the highest power of £ 1 which divides a, if a / 0, 

0 if a ■ 0. 


(a)i 


Note that (a)i Is also 0 If a j 0, but ^ does not divide a. 
For example, 28 - 2**7; so (28) Q «2, (28)i ■ 0, (28)a * 0, 
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( 28)3 ■ 1 , and (28).^ - 0 for any 1 > 3 * 

As a ranges over all non-negative integers, 
range over all m-tuples of natural numbers; and, in fact, as a 

^0 5 m —1 

ranges over 0 to or beyond the functions 

max ((a)o» 5 o) * * * * > max ( (®)m-l # 5 m-i) ran £ e over all m-tuples 


x 0 ' for £q ^ 5o'****^i £ 5m~l* < The functlon (®)i 


couW have been u8ed ln plac * of [a^J and ln treatlne 

the preceding case.) 

Now say E is F*G, where expressions as before are assumed 


already constructed for F and for G. 

u factors 


An occurrence of E is 


an occurrence of F , 7 '^'"''r'FG for some u > 0. But for a 
given £, we must have u £ £, since F and 0 are each of length 
> 1. So now we have for E*F the following: 


a<E) 3 (*>)„< 6 (j,)A<E,n) 

= <5a>n<6( £ ) [<i>l<^max ((n) Q1 E )l(««((n) i » 
P - 2 y J (max ((n) s , ^<£) - ))) 


% 


. , max 
- (n,ax ->max ((n) 0 <£ ) + l'WE) - D. £ 

(max ((n) a , ^ (p) - l)))J , 


where 

6 (£) 


2 + ? (e)+ t?(e) 

Vl 


since each n which would be wanted is of the form 
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Y.i 3L, K 

Ei 2 E 

u 11+1 > 


where 


and 


E < E» 

C /(p) , 

w < ?^(e) > 


max((n) n ,p) , 

Xn) - 2 (max ((n) fll ^(£) - 1)) + 

0 BE JL " mm 

? (max lllU((n| 0 ,£|rt'W - 


It will be seen by readers familiar with recursive function 
theory that occurs ending at time is thus primitive 

recursive in the predicates a*<i>.2k < t) giving the inputs 

over time t « 0 ,though, of course, the recursive 
expressions are complicated. Also, we can express the result 

is primitive recursive in 
p and a number giving in code form the combined input from 


by saying {5 occurs ending at time £ 
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APPENDIX 3 : AN EXAMPLE OF AN EVENT WHICH 13 NOT REPRESENTABLE 

( AND THEREFORE NOT REGULAR) ,■ THOUOH IT IS PRIMITIVE 
RECURSIVE: 


Consider the event E referring to one input cell N, des¬ 
cribed as follows: 

N fires at time t_ - v* for every v such that v* < £, and 
only then. 

(This is primitive recursive, since it can be expressed 


thus: 




No nerve net or finite automaton of any other kind can 
represent this event. For consider an automaton with states 
ai,•..,a r . 

Assume given a representation of the event by a property 
of the automaton at time jaj i.e., we assume that there are 
states, say a*,...,a (r_i < m), such that at time £ the state 
is or is not one of these, according as the event has occurred 
or not. 

We shall show that this assumption leads to absurdity. 

Consider any number £ such that 2£ > ri. 

I Say that N is fired at times t ■ 1, 4, 9*»**,s a and 
never xnereafser. 

Then E occurs for £ « 1, 2,...,(s+l)* — 1 and for 
no greater £. 

Consider the states of the automaton at times s*+l, s a +2, 
... . These must all be from the list a 1; ..,,a . 
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However, beginning with time £ a +l, N is never fired; so 
the external state is constant. Thus, each state for all time 
thereafter will be determined by the immediately preceding 
state. Hence, since there is only a finite number of possible 
states a-i,...,a r , the sequence of the states di,d 2 ,d 3 ,... 
beginning with that at time s. a +l is ultimately periodic. For 
after r states at most, a state must be taken for the second 
time, and thereafter the states since the first occurrence of 
that one must repeat themselves cyclically. 

However, during the time s®+l,... ,s 8 + 2 i 3 , the state must 
be one of a l ,...,a r>i , since the event occurs for these values 

of p; and hence, since 2 s > r 1 , the period must already have 
become established (i.e., the first repetition in di,d 2 ,d 3 ,... 
must already have occurred) by the time s a +2s. Hence the 
state at time s a + 2 s+l (■ (s+ 1 ) 8 ) is one of ai,*..,a r ^, contrary 
to the fact that the event has not occurred ending at time 
£ « (s+ 1 ) 8 . 

It is not suggested that the event in question would be 
of any biological significance. But the example is given to 
show the mathematical limitations to what events can be repre¬ 
sented. Of course, by Appendix 2 we already knew that events 
not primitive recursive are not representable; but the present 
example is much simpler. 

Without either appendix, one would not expect events 
Whose verification Involves the completion of an infinite pro¬ 
cess (these being non—recursive) to be representable. The 
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present example does not Involve the completion of an infinite 
process; but It does involve the completion of a finite process 
which as £ varies is unbounded, and this likewise transcends 
the capabilities of a fixed finite automaton. 
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